emu] 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OMB  No.  0704-01 M 


1*.  REPOST  SECURITY  CLASSIFICATION 
UNCLASSIFIED 


2*.  SECURITY  CLASSIFICATION  AUTHORITY 
N/A 


2b.  DECLASSIFICATION  /DOWNGRADING  SCHEDULE 

N/A 


4.  PERFORMING  ORGANIZATION  REPORT  NUMBER(S) 

RMS-87-R-7 


&*.  NAME  OF  PERFORMING  ORGANIZATION 
Grunman  Corporation 


6c  ADDRESS  (Oty,  S(jf«,  tnd  ZIPC ode) 


lb.  RESTRICTIVE  MARKINGS 

N/A 


3.  distribution/availability  of  report 
Approved  for  public  release;  distribution 
unlimited. 


5.  MONITORING  ORGANIZATION  REPORT  NUMBER(S) 

RADC-TR-87-225 


Bethpage,  Long  Island 
New  York  11714 


8*.  NAME  OF  FUNDING  /SPONSORING 
ORGANIZATION 

Rome  Air  Development  Center 


Sc.  ADDRESS  (City,  SUtt.  tnd  ZIP  Code) 

Griffiss  AFB  NY  13441-5700 


6b  OFFICE  SYMBOL  7a.  NAME  OF  MONITORING  ORGANIZATION 
(If  tpplktble) 

RM&S  Rome  Air  Development  Center  (RBET) 


7b.  ADDRESS  (City,  State,  tnd  ZIP  Cod*) 

Griffiss  AFB  NY  13441-5700 


3b  OFFICE  SYMBOL  9  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 
(If  tppiictblt) 

RBEX  F30602-85-C-0103 


0.  SOURCE  OF  FUNDING  NUMBERS 


PROGRAM  I  PROJECT 

ELEMENT  NO  I  NO 


62702F 


11.  TITLE  ( Include  Security  Cltuificttion) 

IMPROVED  OPERATIONAL  READINESS  THROUGH  EVNIRONMENTAL  STRESS  SCREENING 


12.  PERSONAL  AUTHOR(S) 

Victor  H.  Pellicione,  Joseph  Popolo,  et.  al. 


13a.  TYPE  OF  REPORT  13b.  TIME  COVERED  14.  DATE  OF  REPORT  (Year,  Mor.tft,  Dty)  IIS  PAGE  COUNT 

Final  from  Sep  85  TO  May  87  OeSsbcT  1987  -flenr',  1  3  78 


16.  SUPPLEMENTARY  NOTATION 

N/A 


18.  SUBJECT  TERMS  (Continue  on  reverie  it  necessary  tnd  identify  by  block  number) 
Reliability  Environmental  Stress  Screening 

Quality  Control 


17.  COSATI  CODES 


GROUP  SUB-GROUP 


14  04  07 


12  01  09 


19.  A8STRACT  (Continue  on  reverie  if  necesstry  tnd  identify  by  block  number ) 

The  objective  of  this  study  was  to  develop  guidelines  for  the  application  of  Environmental 
Stress  Screening  (ESS)  to  field  inventory  hardware.  Methods  were  developed  for  the  selection 
of  equipments  for  ESS  application  which  offer  significant  potential  for  operational  readiness 
improvement  and  life  cycle  cost  reduction.  A  2:1  reduction  in  field  removal/maintenance 
action  rates  was  found  for  hardware  subjected  to  ESS  when  compared  against  Identical  hardware 
not  subjected  to  ESS.  A  large  percentage  of  the  improvement  in  the  ESS  hardware  was  found  to 
be  traceable  to  reductions  in  Cannot  Duplicate  (CND)  and  Retest  OK  (RT0K)  rates.  LRU's, 
identified  by  serial  number,  which  demonstrate  recurring  removal  histories  much  higher  than 
population  norma,  were  found  to  make  excellent  candidates  for  ESS.  Tradeoffs  between  screen¬ 
ing  implementation  costs  (facilities,  repair)  and  downstream  logistics  support  savings,  over 
the  remaining  product  life,  were  also  found  to  be  critical  to  the  ESS  candidate  selection 
process.  Recommended  random  vibration  and  temperature  cycling  screening  regimens  are  also 
included  in  the  guidelines .  ~-f  - 


20.  distribution /availability  of  abstract 

BD  UNCLASSIFIED/UNUMITED  □  SAME  AS  RPT.  Q  OTIC  USERS 

21.  AtSTRACT  SECURITY  CLASSIFICATION 

UNCLASSIFIED 

22*.  NAME  OF  RESPONSIBLE  INDIVIDUAL 

Eugene  Fiorentino 

22b.  TELEPHONE  (Include  Are*  Code) 
(315)  330-3476 

OO  Form  1473,  JUN  84 


Previous  editions  ere  oSmW#.-'  V , 


f  CLASSIFICATION  OF  THIS 

UNCLASSIFIED 


omoi&ow 


PREFACE 


This  final  technical  report  was  prepared  by  Grumman  Aircraft  Systems  Division!  ~ 
(ASD) ,  Reliability,  Maintainability,  and  Safaty- Section,  Bethpage,  New  York,  under 

V  '*  " 

contract  F30602-85-C-0103  for  Rome  Air  Development  Center  (RADC).  Mr.  Eugene 
Fiaramir.o  was  the  project  engineer  for  RADC. 

The  effort  described  was  accomplished  during  the  period  from  September  1985 
through  May  1987. 

Under  the  direction  of  principal  engineer  Mr.  Joe  Popolo,  the  Grumman  study 
team  included  project  engineer,  Victor  Pellieione,  technical  staff;  Jeff  Triolo,  Murry 
Rosenfeld,  Brian  Farrell,  Karl  Rehm,  and  Arlene  Apollo. 

We  wish  to  further  acknowledge  the  field  interface  support  of  Mr.  John  "Dusty" 
Rhoades  of  Grusu.:an's  Field  Marketing  Department,  and  the  cooperation  of  the  nu¬ 
merous  Air  Force  logistics  centers  and  operational  facilities  visited  during  the  study 
including: 

•  Wright-Patterson  AFLC 
Wright- Patterson  AFB,  Ohio 

•  Warner  Robins,  ALC 
Robins,  AFB,  Georgia 

•  Oklahoma  City.  ALC 
Tinker  AFB,  Oklahoma 

•  Sacramento,  ALC 
McClellan  AFB,  California 

•  Langley  TAC/HQ 
Langley  AFB,  Virginia. 


By 

Ci:  t  ib  itior.  I 


Av.tilnL1iiv  C.xJcs 


EXECUTIVE  SUMMARY 


Environmental  Stress  Screening  iESSl  is  currently  being  used  extensively,  by 
electronic  equipment  manufacturers,  as  a  means  for  effectively  removing  process 
induced,  v/orkmanship  and  part  defects  from  their  products.  Much  of  the  current  Ai - 
Force  electronic  equipment  inventory  has  not  had  exposure  to  ESS.  Questions  and 
issues  regarding  the  application  of  ESS  to  field  inventory  hardware  are  addressed  in 
this  study.  Guidelines  are  developed  for  cost-effectively  applying  ESS  to  field 
inventory  hardware. 

Electronic  equipment  currently  in  the  operational  inventory  contain  residual 
latent  defects  wh’ch  were  introduced  into  the  hardware,  either  during  the  original 
manufacturing  process  or  through  prior  maintenance  handling  and  use  operations. 
Latent  defects  are  weaknesses  or  flaws  in  parts  and  connections  which  exhibit  much 
higher  failure  rates  du^'ng  early  operational  life  than  reliability  design  goals 
would  indicate.  Such  defects  manifest  as  functional  intermittent*  resulting  ^n 
excessive  Cannot  huplicate  fCMDl  and  Retest  OK  fRTOKl  actions,  as  well  as  hard 
failures.  Latent  defects  are  in  effect  usage  and  process  induced  and  not  functions 
of  the  inherent  design  life.  It  is  highly  l’kely  that  selective  application  of  ESS 
to  such  equipment,  under  properly  controlled  conditions,  will  result  in  significant 
improvements  in  field  reliability. 

Field  maintenance  data,  representing  over  10  years  of  field  operational  and 
maintenance  history,  were  used  to  assess  the  effectiveness  of  ESS  in  improving  field 
reliability.  Five  case  histories  of  electronic  LRUs  of  various  complexity  and  design 
were  compiled  and  analyzed.  The  LRU  case  history  populations  contained  some  LRUs 
which  were  exposed  to  ESS  in  the  manufacturer’s  plant  and  others  which  were  not.. 
Comparison  of  the  ESS  vs  non-ESS  populations  provided  the  means  for  assessing  the 
effectiveness  of  ESS,  under  actual  field  stress  conditions  of  operation  and  use.  The 
Improvements  achieved  in  terms  of  Mean  Flight  Hours  Between  Removal  (MFHBR1  of  the 
ESS  versus  the  non-ESS  populations,  are  shown  in  Table  1. 


TABLE  1 

RELIABILITY  IMPROVEMENT  -  ESS  VS  NON-ESS  EQUIPMENT 


EQUIPMENT 

NON-ESS 

MFHBR 

ESS 

MFHBR 

- % 

IMPROVEMENT 

A 

32 

70 

119 

B 

66 

127 

92 

C 

83 

80 

D 

92 

202 

E 

570 

■hI 

95 

The  data  clearly  demonstrate  the  benefits  of  ESS. 

o  ESS  of  new  hardware  M.e.  at  a  manufactu-er's  facility!  Is  effective, 
o  Reduction  of  overall  removal  -ates  results  In  improved  MFHBR  on  the  order  o* 
2:1  across  the  board. 

Economic  analysis  of  the  case  history  data  <nd*cate  that  significant  savings  *n 
logistic  support  cost,  ranging  in  the  millions  of  dollars,  were  realized,  as  shown  in 
Table  2. 


TABLE  9 

LOGISTICS  SITPORT  COST  SAVINGS  -  ESS  vs  MON-ESS  E0UIBMENT 


TOTAL  LSC 

LSC 

nr  lru 

AVQ 

AVQ 

AVQ 

DIRECT 
LSC  W/O 

DIRECT 
LSC  W 

DELTA 

DELTA 

SAVINGS  AS  A 

UNIT 

MFHBR 

MFHBR 

ESS 

ESS 

SAVINGS 

SAVINGS 

*  OF  LRU 

LRU 

8 

W/O  ESS 

*  ESS 

(M 

SM 

SM 

8K 

UNIT  COST 

A 

312.000 

32 

70 

151.7 

57.7 

94.0 

261.1 

84 

8 

82,038 

86 

127 

26.4 

11.5 

14.9 

418 

50 

C 

162,932 

83 

149 

48.0 

251 

20.9 

58.1 

36 

0 

28.180 

92 

278 

9.2 

68 

3.7 

10.3 

39 

E 

55,217 

670 

1110 

3.0 

18 

18 

4.2 

8 

MOTES: 

•  AF  LSC  MODEL,  VEF'SION  1.1,  JANUARY  1979;  OPERATIONAL  DATA  IN  APPENDIX  A 

•  1985  DOLLARS 

•  SERVICE  LIFE:  18  YEARS _  - 


The  benefits  to  be  derived  f-om  screening  inventory  hardware,  ontentially,  can 
be  at  least  the  same  or  better  than  that  illustrated  by  the  ESS  case  history  data. 
Several  important  differences  should,  however,  be  noted,  with  inventory  hardware, 
actual  field  reliability  performance  can  be  observed  and  used  in  selecting  candidates 
with  the  best  potential  for  improvement.  Therefore,  100%  screening,  as  is  the  case 
for  factory  applied  ESS,  is  not  necessary  nor  cost-effective.  In  addition,  applica¬ 
tion  of  ESS  in  the  field  must  be  performed  in  a  non- homo oeneous  product  control 
environment.  Service  life  build-up  and  repeated  repair  of  field  «quioment  will  have 
affected  lot  homogeneity  as  well  as  identifiable  configuration  controls.  Screen 
implementation  and  inventory  management  costs  will  also  exact  its  toll  on  any  loqis- 
tics  cost  savings  that  may  b“  realized.  For  these  reasons,  a  field  ESS  prooram  must 
be  defined  that  will  identify,  select,  and  optimally  screen  only  those  equipments  and 
levels  of  assembly  which  will  provide  the  most  potential  for  reducing  the  aggregate 
removal  rates,  within  realistic  cost  constraints  and  the  sho-t^st  t'‘me  practical. 

Comparison  of  the  ESS  and  non-ESS  case  V story  data  provide  several  significant 
findings  which  form  the  basis  for  the  ouideline  development.  The  scope  of  the  study 
data  base  is  shown  in  Table  3. 


TABLE 

ESS  VS  MON-ESS  CASE  HISTORY  HATA  RASE 


" 

NON-ESS 

6SS 

EQUIP- 

TOTAL 

HQ.  a;c 

TOTAL 

NO. 

FLIGHT 

VM 

TOTAL  NO. 

UNITS 
REPORTING 
(SER  NO. 

TOTAl  NO. 

removals 

AVG.NO. 

REMOVALS/ 

TOTAL 

NO.  A/C 

TOTAL 

NO. 

FLIGHT 

VRI 

TOTAL  NO. 

UNITS 
REPORTING 
tSCR  NO. 

TOTAL  NO. 
REMOVAL! 

AVG.NO. 

REMOVALS. 

MENT 

REPORTING 

HRS 

OPERATION 

LRUs) 

FOR  CAUSE 

UNIT 

REPORTING 

HRS 

OPERATION 

LRUO 

FOR  CAUSE 

UNIT 

A 

222 

3 20.451 

11 

110€ 

11895 

to.s 

4 14 

120.140 

4 

612 

mo 

21 

i 

222 

110.432 

II 

110 

MU 

SI 

114 

120.140 

4 

461 

Ml 

II 

C 

22? 

110.412 

11 

•40 

4445 

1.2 

114 

I20.KO 

4 

412 

106 

11 

D 

221 

120.411 

II 

1071 

4011 

3.1 

»*.< 

120.140 

4 

:» 

411 

1.» 

i 

41 

121.601 

‘ 

111 

1330 

3.1 

— 

n 

31,112 

i 

1SS 

211 

11 
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The  findings  include: 


o  level  of  Assembly  Testing  -  The  LRU  is  denned  as  the  focal  po'nt  for 
hardware  selection  and  sc*“een*ng.  In  a  f*eld  scenario,  it  is  the  collects* 
vehicle  for  all  quality  defects.  Screening  the  LRU  subjects  the  entire 
population  of  assemblies  and  pa**t.s  to  the  screen.  The  LRU  provfdes: 

Measurable  removal  ra+es  obtained  under  actual  operating  stress 
condi tions. 

A  means  for  monitoring  and  selecting  individual  units  by  serial  numbe-. 
The  basis  for  "Bad  Actor"  hardware  selection. 

By  contrast,  there  is  no  significant  maintenance  improvement  or  cost  benefit 
that,  can  be  justified  by  indiscriminate  screening  of  SRUs  and  lower  levels  of 
assembly.  One-at-a-tiire  repair  by  replacement  with  screened  lower  levels  of 
assembly  has  little  ’’moact  on  the  aqgregate  LRU  defect  nopulat/on.  To  have 
any  impact  at  all,  the  lower  level  of  assembly  must  make  a  sionificant 
contribution  to  tbe  LRU  removal  r3te,  ano  the  removal  -ate  of  the  LRU  must  be 
significantly  high  in  order  to  have  the  greatest  impact  on  operational 
readiness. 

0  Uardware  Selection  -  In  the  selection  process,  high  contributor  removal 
rates  will  provide  the  prioritization  for  initial  LRU  selection.  "Bad  Actor" 
selection  offers  the  best  potential  for  attacking  the  highest  percentage  of 
ESS  sensitive  defectives,  wftMn  a  small  percentage  of  the  total  LRU 
populat'‘on.  The  serial iz.ation  of  LRUs  and  their  traceability  provides  a 
process  control  mechanism  for  identifying  and  determining  the  distribution 
of  removals  by  individual  unit.  Serial  number  tracking  enables  identifica¬ 
tion  of  specific  units  ("Bad  Actors"!  baying  maintenance  act! on /removal  fre¬ 
quencies  much  higher  than  the  operational  norms.  Tn  tbe  case  bistories 
studied,  Bad  Actor  ranks  constituted  more  than  70%  of  the  removals  in  less 
than  30%  of  the  units.  For  the  non-ESS  equipment,  more  than  SOt  of  the 
removals  were  false  alarms  or  cannot  duplicate  actions  which  are  known  to  be 
highly  correlated  to  ESS  detectable  intermittent  type  failures. 

o  Generic  ESS  -  In  the  implementation  ef  the  formal  FS3  test  program  at  the 
ALC,  generic  ESS  profiles  for  rapid  thermal  cycles  and  random  vibration  are 
defined  and  reconwended.  In  each  of  the  case  histories  generic  practices 
were  used.  Generic  screening  profiles  will  simplify  ESS  testing  operations, 
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reduce  ESS  set-up  costs  and  irrMnv’ze  CSS  traWnp  requirements.  The  aene^'c 
guidelines  are  typical  of  those  defined  by  MIL-STO-’IM  fEO  and  MAVMAT  P- 
949?.  In  addition,  guidelines  for  tailoring  are  provided  for  those  cases 
where  the  equipment  may  not  have  been  designed  to  ^unction  wit.h’'n  the  defined 
random  vib-ation  or  rapid  thema  evening  environments.  H$e  of  the 
techniques  will  minimize  the  potential  for  overstressing  the  equipment  when 
generic  ESS  levels  are  applied. 

o  ESS  Economics  -  The  economics  of  field  ESS  dictate  that  the  optimal  Return-on 
Investment  (ROI,  nominally  S3  1/3%1  is  achieved  most  consistently  by 
screening  only  a  select  minimum  number  of  units.  Bad  Acto-  selection  offers 
the  best  opportunity  to  achieve  this  goal.  To  minimize  the  cost  henefit 
risk: 

Selected  units  should  he  high  removal  rate  contributors,  coupled  with 
high  unit  cost.  This  will  maximize  the  logistics  support  cost  savinos. 
Screen  durations  should  be  reduced,  where  possible,  to  minimi/e 
screen/test  facility  loading  and  lower  screen  implementation  costs. 
Lower  levels  of  assembly  screening  (SRU'  should  demonstrate  significant 
improvement  potential  in  LRU  removal  rate,  nominally  greater  than  ?5*, 
to  ensure  effective  ROI  payoff. 

Screening  of  aged  or  low  removal  rate  equipment  should  always  be  justi¬ 
fied  on  a  ROI  basis. 

In  order  to  assist  cognizant  Air  Force  Legists  Center  f ALC '  and  responsible 
item  managers  in  planning  and  implementing  an  ESS  program  for  field  inventoried 
equipment,  guidelines  are  provided  herein  as  Appendix  C.  The  guidelines  provide  the 
methodology  for; 

1.  Equipment  Selection  -  Based  on  established  equipment  field  maintenance 
histories. 

2.  Selection  Criteria  -  Designed  to  minimize  the  quantity  of  equipment  selected 
for  screening  and  to  maximize  improvement  in  operational  readiness. 

3.  Test  Profile  Development  -  Designed  to  exercise  established  and  proven  temp¬ 
erature  and  vibration  screening  levels  and  durations  within  equipment  design 
qualification  limits  and  operational  capability. 

4.  Economic  Selection  -  Designed  to  minimize  cost  of  testing  and  to  maximize 
logistics  support  cost  savings  withir.  realistic  Return  on  Investment  (ROI' 
goals. 


Future  study  efforts  should  focus  on  validating,  ref*n*ng  and  standardizing  the 
approach.  We  recommend  that: 

o  Severa1  pi  lot  study  projects  he  undertaken  on  inventoried  equ’oment  to 
thoroughly  evaluate  the  guideline  nethodoloo/. 
o  Bad  Actor  selection  offers  t.h«  opportunity  to  select  a  small  number  of  LR'ls, 
which  minimize  the  Impact  on  IRIJ  availability  and  system  readiness,  and  vet 
effectively  reduces  a  High  onrcentag^  of  the  ESS  sens’’tive  removal  rat.e.  The 
pilot  projects  should  bo  used  to  validate  and  refine  the  sel°ction  r*"'te-ia, 
scneen'ng  and  cost  benefit  methodology  contained  in  t.be  ouideline. 
o  Ai»*  Force  field  maintenance  data  and  performance  monitoring  systems  should 
be  modified,  where  necessary,  to  enabla  serial  number  tracking  of  the  hard¬ 
ware  fLRUs  and  SRUs'  and  to  facilitate  application  of  ESS  to  all  high  removal 
rate  products  *n  the  inventory. 
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1  -  INTRODUCTION 


Application  of  Environmental  Stress  Screening  (ESS)  during  manufacture  of 
electronic  equipment  is  attaining  widespread  use  as  the  means  for  reducing  the  ef¬ 
fects  of  manufacturing,  quality,  and  system  process  defects  on  field  performance. 
These  defects,  commonly  termed  "latent  defects",  are  traceable  to  poor  workmanship, 
out-of-control  processes,  or  defective  parts  and  assembly.  Analyses  of  field  data 
have  shown  that  latent  defects  can  have  a  severe  impact  on  the  excessive  removal 
rates  of  hardware  in  the  field,  resulting  in  reduced  field  reliability  and  weapon  sys¬ 
tem  readiness. 

Further,  hardware  subjected  to  many  repair  cycles  or  to  field  modification  and 
handling  may,  as  a  result  of  these  actions,  have  additional  defects  induced  which 
exacerbate  the  hardware's  degradation.  These  defects  can  be  introduced  by  spare 
parts  and  lower  levels  of  assembly  which  have  not  been  previously  screened,  and  by 
poor  workmanship  and  maintenance  practices  during  repair  and  handling.  The 
defects  manifest  as  excessive  removal  rates  caused  by:  functional  intermitten ts 
which  precipitate  false  alarm  (or  no  defect  found)  actions,  excessive  tolerance  and 
functional  checks,  as  well  as  repairs  due  to  broken,  loose,  or  mismatched  compo¬ 
nents  . 

Hardware  which  has  not  had  ESS  exposure  can  exhibit  much  higher  removal 
rates  during  early  or  even  sustained  operational  life  than  predicted  or  demonstrated 
reliability  baseline  values  would  indicate.  Since  defects  are  in  effect  attributes  of 
specific  equipments  and  not  a  function  of  the  inherent  design  life,  there  is  the  pos¬ 
sibility  that  the  application  of  ESS  to  such  equipment,  under  controlled  conditions 
can  result  in  significant  improvements  in  field  performance. 

Conversely,  on  a  theoretical  basis,  hardware  that  has  been  in  the  field  for  any 
length  of  time  might  have  had  latent  defects  precipitated  to  failure  purely  by  the  en¬ 
vironmental  stresses  naturally  imposed  by  the  field  conditions.  Under  these  circum¬ 
stances,  field  ESS  application  would  provide  only  marginal  improvement  in  the  field 
reliability  performance. 


Application  of  ESS  to  all  inventory  hardware  is  obviously  not  practical  nor  cost- 
effective.  This  report  provides  the  basis  for  selectively  applying  ESS  to  field  in¬ 
ventoried  electronic  hardware.  Significant  potential  is  offered  for  improving  field 
reliability  and  thereby  the  operational  readiness  performance  of  Air  Force  weapon 
systems  as  well  as  for  large  reductions  in  maintenance  and  support  costs.  This  re¬ 
port  analyzes  all  aspects  of  the  problem,  from  both  test  effectiveness  and  cost  view¬ 
points,  and  provides  guidelines  for  cost-effective  implementation  of  ESS  in  the  field 
maintenance  environment. 


2  -  APPROACH 


2.1  BACKGROUND 

In  general,  the  classical  field  maintenance  removal  rate  improvement  of  an  elec¬ 
tronic  equipment  due  to  ESS  can  be  depicted  as  shown  in  Fig.  1.  The  primary  ob¬ 
jective  is  to  minimize  the  total  removal  rate  which  consists,  theoretically,  of  both 
latent  (workmanship,  ESS  sensitive  defects)  and  inherent  (design  life)  defects,  to  at 
least  some  improved  level  which  is  closely  allied  to  the  specified  or  operational 
baseline  goals. 
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Figure  1.  Typical  removal  rata  improvement  effect*  due  to  ESS. 


The  removal  rate  and  its  time  distribution  for  a  weapon  system  and  its  Line  Re¬ 
placeable  Units  (LRUs)  is  obtained  via  the  typical  field  maintenance  data  and  report¬ 
ing  system  (Air  Force  D056,  Navy  Maintenance  and  Material  Management  (3M),  Army 
UMSDC).  A  properly- designed  ESS  profile  (temperature  and  vibration)  test  will  ef¬ 
fectively  reduce  the  latent  defect  population,  resulting  in  a  gain  from  a  non-ESS  re¬ 
moval  rate  to  an  ESS  rate.  The  difference  or  improvement,  therefore,  should  be  a 
function  of  the  latent  defect  rate  reduction,  with  the  resulting  rate  residue  being 
mostly  inherent  in  nature.  It  is,  however,  difficult  to  demonstrate  such  a  scenario 
with  any  clear  conviction  in  a  real-world  field  maintenance  environment  since,  the 
field  data  do  not  always  contain: 

•  Clear  definition  of  failure  or  effect 

•  Laboratory  failure  analysis  to  classify  latency  or  inherency  or  other 

•  Consistent  maintenance  practices  and  reporting 

•  A  closed- loop  traceability  of  actions  from  LRU  to  piece  part  without  voids  in 
the  physical  repair  process,  and  hardware  identities 

•  Accurate  diagnosis  without  a  multiplicity  of  no- fault  removals  for  unknown 
reasons 

•  Time  in  service,  utilization  and  operating  hours,  and  power-on  time  per 
equipment  is  at  best  a  function  of  the  weapon  systems  service  time  (e.g., 
aircraft  flight  hours,  system  ownership  time). 

Further,  the  specified  rate  which  may  be  defined  by  handbook  predicting  techniques 
(e.g.,  MIL-HDBK-217),  specified  contractual  goals,  or  field  operational  objectives 
(e.g.,  R&M  2000  targets),  represents  purist  values  and  is  not  quantitatively 
measurable  from  field  maintenance  data  for  the  above  reasons.  This  results  in 
measured  comparisons  that  are  clouded  by  definitions  and  groundrules.  Therefore, 
the  effect  of  meeting  or  exceeding  a  specified  value  by  groundrule,  without  showing 
improvement  in  the  field  removal  rate  is  almost  meaningless  from  a  logistics,  readi¬ 
ness,  and  ultimate  life  cycle  cost  point  of  view. 

2.2  APPROACH 

Figure  2  outlines  the  task  flow  of  analyser  -nd  assessments  performed  during 
this  study  and  described  in  the  subsequent  sections  of  this  report.  A  Field  ESS 
Implementation  Guideline  is  also  provided  in  Appendix  C.  The  basis  of  the  approach 
is  to  establish  that  the  differences  in  removal  rates  can  be  realized  first  by  ESS  in 
general,  and  then  by  incorporating  a  planned  ESS  program  in  a  field  maintenance 
environment  at  an  Air  Logistics  Command  (ALC)  facility. 
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Figure  2.  Study  operational  task  flow  for  ESS  selection. 


To  accomplish  this,  five  case  histories  of  electronic  LRUs  of  varying  complexity 
and  design  were  selected  from  complex  tactical  weapon  systems.  Initially  each  of  the 
LRUs  did  not  have  ESS  exposure  during  their  early  procurement,  either  at  a  vendor 
or  field  facility,  and  each  had  established  long  histories  of  field  performance.  As  a 
result  of  product  improvement  in  later  years,  ESS  requirements  were  introduced  for 
subsequent  production  lot  acceptance,  which  provided  a  similar  long  history  oi  an 
ESS  population  that  could  be  directly  compared  to  their  non-ESS  counterparts. 
These  data  provided  the  means  for  assessing  the  effectiveness  of  ESS  under  real 
world  conditions  of  maintenance  and  use  as  described  by  the  field  maintenance  data. 


The  effects  of  screening  field-aged  hardware  were  assessed  using  experience 
data  and  engineering  judgements.  Trade-offs  between  screening  implementation  costs 
(facilities,  test,  repair)  and  downstream  logistic  support  cost  savings  over  the  re¬ 
maining  product  life  are  a  critical  selection  factor.  An  economic  analysis  procedure 
was  developed  and  is  presented  on  a  cost  benefit  Return  On  Investment  (ROI)  basis. 
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3  -  FIELD  DATA  ASSESSMENT 


A  field  data  assessment  was  performed  to  study  the  effectiveness  of  ESS  as  a 
function  of  actual  field  operational  performance.  The  assessment  was  accomplished 
by  selecting  five  avionic  equipment  types  which  had  previous  field  history  as 
non-ESS  equipment,  and  as  a  result  of  product  improvement,  had  subsequent  pro¬ 
duction  lots  subjected  to  ESS  as  part  of  the  acceptance  test  requirements  at  the 
manufacturer's  facility.  All  screened  units  were  tested  as  new,  and  not  previously 
field- deployed  or  overhauled.  Figure  3  and  Table  1  provide  the  identification,  de¬ 
scription,  physical  statistics  and  features  of  the  LRUs  selected.  The  five  equip¬ 
ments  were  conceived  (in  design)  late  in  1960  and  introduced  to  the  field  in  early 
1970.  The  equipment  field  performance  and  maintenance  history  spans  six  to  eleven 
years  for  the  non-ESS  population,  and  four  to  five  years  for  the  ESS  population; 
thereby  providing  a  sound  maintenance  data  base  for  making  comparisons. 

All  of  the  equipments  have  field  maintenance  histories,  as  formally  reported  in 
the  Navy's  3M  reporting  system.  The  data  was  received  in  raw  magnetic  tape  format 
for  processing  in-house,  on  a  monthly  basis.  A  continuous  updating  of  the  histor¬ 
ical  databases,  thus  provided  as  complete  a  field  maintenance  history  as  possible. 


3.1  EQUIPMENT  SELECTION 


The  equipments  selected  for  this  study  are  currently  in  operation  use  on  two 
aircraft  weapons  systems,  the  F-14  and  E-2C.  As  previously  noted,  the  ESS  units 
in  all  cases  were  ESS  tested  at  the  vendor  facility  and  delivered  with  the  weapon 
system  as  new,  and  not  as  units  puiied  from  the  field  and  then  subjected  to  ESS.  The 
configuration  of  the  ESS  units  are  very  similar  to  the  non-ESS  units  except  for  some 
minor  engineering  changes/modificaiions,  which  is  considered  a  natural  process  over 
a  ten  year  period.  ESS  LRUs  were  subjected  to  a  formal  ESS  test  which  consisted  of 
rapid  thermal  cycling  and  random  vibration  similar  to  those  defined  by  NAVMAT- 
P9492  and  MIL-STD-2164(EC)  (Ref  1  and  2).  The  specified  ESS  test  characteristics 
for  each  equipment  are  summarized  in  Table  2  and  discussed  more  thoroughly  in 
Section  5.  Data  generated  for  this  study  spanned  an  11  year  period  (1975-1985)  for 
non-ESS  units  aud  five  years  (1981-1935)  for  the  ESS  units. 
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8  -  CENTRAL  AIR  DATA  COMPUTER  (CAOC) 

Figure  3.  Physical  futures  of  selected  equipments  (sheet  1  of  2). 
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Figure  X  Physical  features  of  ttioctod  aquifKiMtits  (shaat  2  of  2). 


TABLE  1.  LRU  characteristics. 


LRU 

IRIEF  DESCRIPTION 

WEIGHT 

(LBS) 

FORM  FACTOR 
UxWxHI 

IN  INCHES 

SPECIFIED 

MTCF 

(HRS) 

NO. 

SRIh 

NO. 

HYSRIOS/ICs 
(%  OF  TOTAL 
PARTS) 

OTHER  PARTS 
IK  OF  TOTAL 
PARTS) 

TOTAL 

NO.  PARTS 

QUANTITY 

PER 

WEAPON 

SYSTEM 

A  tCSDC) 

COMPUTER 

SIGNAL  OATA 
CONVERTER 

PHOVIOES  SIGNAL  TIMING, 
FORMATTING,  SWITCHING, 
GENERAL  PURPOSE  COM* 
PU1ATI0NAL  CAPABILI¬ 
TIES,  4  INTERFACE 
COMPATIBILITY  BETWEEN 
ASSOCIATED  AVIONICS 
EQUIPMENT 

41.7 

13.75  x  12.82  xS.68 

■ 

420 

49 

995  143) 

1325  (57) 

2320 

1 

a  icaoc) 

CENTRAL  AIR 

OATA  COMPUTER 

PROVIDES  AIR  DATA 
FUNCTIONS  FOR  AIRCRAFT 
AVIONIC  SYSTEMS 

33’ 

21.1  x  8.7  it  6.7 

2070 

21 

202  111) 

735 1  (89) 

2643 

1 

C  (VOIG) 

VISUAL  D'SPLAY 
INDICATOR 

GROUP 

(PROCESSOR! 

RECEIVES  INPUTS  FROM 
VARIOUS  AIRCRAFT  SYS¬ 
TEMS  &  PROVIDES  ANALOG 
OUTPUT  SUITABLE  FOR 
GENERATING  ANALOG 
DISPLAY  INDICATOR 
SYMBOL  OiSPLAYS 

56.1 

25.0  x  10.4x9.4 

202 

51 

2704  ISO) 

2711  (501 

5415 

1 

0  (AICS) 

AIR  INLET 
CONTROL  SYSTEM 
(PROGRAMMER) 

PROVIOESTHE  CONTROL  ! 
&  FAILURE  MONITORING 
FUNCTIONS  OF  AIRCRAFT 
AIR  INLET  CONTROL 
SYSTEM 

14.0 

16.0  *4.0  x  7.0 

JI700 

! 

«  i 

IB9II2) 

1257  (88) 

1426 

2 

E  (SCRAM) 

SIGNAL  COMMAND 
READOUTS 
ALARM  MODULE 

PROVIDES  INTERFACE  BE¬ 
TWEEN  COMPUTER  PRO¬ 
GRAMMER  'CPI  a  AVIONIC 
SUBSYSTEMS  BEING 
MONITORED 

80 

10.0x5x6.7 

5000 

1 

139  151) 

136  (491 

275 

6 
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TABLE  2.  ES3  test  characteristics. 


LRU 

j 

THERMAL  CYCLING 
T/C 

VIBRATION 

BEFORE  &  AFTER  T/C 

NO.  OF  CYCLES  j 

CYCLE 

DURATION 

(HRS) 

BURN-IN 

FAILURE 

FREE 

A 

-54°  TO+71°C@ 
5°C/MIN. 

•  RANOOM  -4.9  GRMS 

FOR  15  MIN. 

•  SIN  -  1,5  G  FOR  10  MIN. 
OF  EACH  HR  ON  TIME 
(35  HRS! 

18 

34 

5.5 

B 

-5*°C  TO  +71°C  @ 
5°C/MIN. 

•  SIN  -  1.5  G  FOR  10  MIN. 

OF  EACH  HR  ON  TIME 
(17  HRS) 

20 

20 

4.0 

c# 

-54°CT0  7fC@ 

5”  C/M  IN 

•  RANOOM  -  5,2  GRMS 

FOR  10  MINUTES 

•  SIN  -  1.5  G  FOR  10  MIN. 
OF  EACH  HR  ON  TIME 

110  HRS) 

10 

20 

4.0 

0 

-54aC  TO  +71°C  @ 
5°C/MIN. 

•  SIN  -  1.5  G  FOR  10  MIN. 
OF  EACH  HR  ON  TIME 
I14HRSI 

5 

32 

3.75 

E 

-40°CTO+553C@ 

S*  C/M  IN 

•  RANDOM  -  6.06  GRMS 
FOR  15  MIN. 

35 

Hi 

1.75 

•DENOTES  THAT  THE  PANDOM  VIBRATION  PORTION  OF  THE  ESS  TEST  IS  APPLIED  TO  1  OUT  OF  EVERY  9  LRU«  | 

AS  A  MINIMUM  | 
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Table  3  presents  all  five  case  histories  with  all  the  associated  statistics,  drawn 
m  the  3M  database.  As  note*,  a  significant  number  of  units  reporting  (different 
ialized  LRUs)  are  identified  in  each  category,  providing  a  sound  statistical  base. 

TABLE  3.  Casa  histories  —  removal  reporting  statistics 

I  ' 


— 

TOTAL  NO 

[ - 

TOTAL  NO. 

- ' 

TOTAL 

1 

UNITS 

TOTAL 

UNITS 

I 

TOTAL 

NO 

REPORTING 

TOTAL  NO 

AVG.  NO. 

TOTAL 

NO. 

REPORTING 

TOTAL  NO. 

AVC.  NO 

FLIGHT 

YRS 

iSER  NO. 

REMOVALS 

REMOVALS/ 

NO.  A/C 

FLIGHT 

YRJ 

(SER  NO. 

REMOVALS 

REMOVALS/ 

REPORTING 

HRS 

0FC  RATION 

LRUiJ 

— 

FOR  TAUSE 

UNIT 

MFHBR 

REPORTING 

HRS 

Pff  RATION 

IRUil 

FOR  CAUSE 

UNIT 

UFHBR 

222 

370.437 

11 

1106 

11695 

10.5 

32 

!M 

120.140 

4 

587 

1718 

2.9 

70 

222 

370.437 

11 

970 

5631 

51 

66 

114 

120.140 

4 

453 

949 

2.1 

127 

222 

370  437 

11 

M9 

4445 

52 

S3 

114 

120.140 

4 

432 

805 

1.9 

149 

1 

222 

370.43  7 

11 

1029 

4031 

39 

92 

114 

170.140 

4 

285 

433 

1.5 

278 

_ 

4? 

126.667 

6 

377 

1330 

35 

570* 

22 

53.157 

. 

S 

151 

717 

1.8 

mo* 

/G  I  UNITS  PER  AjC 
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2  FIELD  DATA  REDUCTION 


The  intent  of  Field  Data  Reduction  is  to  glean  from  the  maintenance  data  system 
ose  indicators  and  parameters  which  can  be  used  for  tracking  field  maintenance 
•rformance,  and  to  assist  in  the  selection  of  candidate  equipments  or  groups  of 
luipment  that  could  be  sensitive  to  corrective  ESS  in  the  field.  Figure  4  illustrates 
ie  process  used  to  categorize  the  data  as  derived  from  the  maintenance  and  repair 
,'ocess . 

In  reducing  Navy  3M  data,  strict  compliance  with  Air  Force  maintenance  (D056) 
:tion  definitions  and  how  malfunction  coding  were  maintained  to  ensure 
.  lat  action  taken  traceability  provided  for  final  disposition  of  the  maintenance  action. 

All  of  the  Job  Control  Numbers  (TCN)  generated  are  associated  with  un- 
;heduled  type  maintenance  (TM  code  =  B)  and  action  taken  (AT)  codes  P  (removed) 
rad  R  (remove  and  replace)  as  per  AFR-300-4  (Vol  3).  Selection  of  these  parame- 
>rs  permits  us  to  focus  on  the  removals  for  cause  maintenance  actions  and  eliminates 
rae  remove  for  access,  cannibalization,  and  unrelated  actions  not  bearing  on  the  re- 
air  disposition  of  the  hardware.  This  refinement  of  the  field  data  presents  a 
lean-cut  accountability  for  the  pertinent  maintenance  actions,  which  is  essential  for 
he  ESS  Effectiveness  Analysis  discussed  in  the  subsequent  sections  of  this  report. 

The  Air  Force  D056  Maintenance  Data  Collection  System  contains  three  mainte- 
lance  categories  as  defined  by  AFR  800-18  and  identified  in  Fig.  4: 
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Ftgur*  4.  Field  data  reduction. 


(1)  Type  1  Inherent  Malfunctions  -  activity  resulting  from  malfunctions  that 
occur  from  internal  design  and  manufacturing  characteristics.  This  < 
egory  consists  of  both  repair  with  definable  part  or  component  assemb 
and  repair  without  parts,  e.g.,  inadequate/exceasive  solder,  loose  pai 
etc.  Repair  with  parts  consist  of  those  removal  devices  e.g.,  LRUs  ('14' 
card  at  ’O’  level)  requiring  the  removal/replacement  of  SRUs  ('14'  card 
T  level)  and/or  direct  removal  of  piece  parts  ('P*  card) 

(2)  Type  2  Induced  Malfunctions  -  activity  resulting  from  malfunctions  that 
occur  from  external  sources,  e.g.,  Improper  maintenance,  operator  en 
foreign  object  damage,  failures  due  to  malfunction  of  associated  equ 
ment,  etc 


(3)  Type  6  No  Defects  -  activity  resulting:  from  malfunctions  which  could  not 
be  Confirmed  nor  Duplicated,  (CND),  e.g.,  removals  which  subsequently 
bench  check  satisfactory,  and  is  reported  on  How  Malfunction  (HM)  code 
799  (no  defect)  only. 

3.3  EQUIPMENT  OPERATIONAL  PERFORMANCE 

In  tracking  the  equipment's  field  maintenance  performance,  strict 
attention  was  paid  to  LRU  serial  number  identification  and  traceability.  Without  the 
traceability  and  reporting  of  LRU  serial  numbers,  the  task  of  segregating  the  non- 
ESS  from  the  ESS  population  would  have  been  almost  impossible.  SRU  and  lower 
level  assemblies  were  in  no  way  traceable  or  accountable.  The  basis  of  operating 
time  in  all  cases  was  flight  hours  as  accumulated  by  aircraft  block  numbers  known  to 
contain  non-ESS  or  ESS  equipment.  Specific  blocks  of  hardware  serial  numbers  were 
assigned  to  specific  blocks  of  aircraft.  By  continuously  cross -matching  the  blocks 
on  a  monthly  basis,  all  removals  and  all  flight  hours  were  accounted  for  in  each 
non-ESS  and  ESS  category.  Where  any  mixing  or  mismatching  appeared,  the  data 
was  eliminated  completely. 

Figure  5  provides  the  annual  historical  trends  of  each  case  history  equipment 
for  both  the  non-ESS  and  ESS  populations.  The  bar  charts  are  expressed  in  terms 
of  removals  per  1000  flight  hours,  and  provide  the  distribution  of  Type  1,  Type  2, 
and  Type  6  actions  for  each  population.  The  only  common  ground  noted  on  this  ba¬ 
sis  is  the  consistent  reduction  in  the  removal  rate  pattern  of  the  ESS  populations. 

Figure  6  provides  the  summary  of  the  statistics  developed  for  each  of  the  popu¬ 
lations  and  the  resulting  rate  improvements  derived  from  the  difference  between  the 
non-ESS  and  ESS  groups.  It  is  apparent  from  these  comparisons  that  a  significant 
improvement  has  been  manifested  across  the  board  for  all  equipments,  with  the  over¬ 
all  removal  reduction  ranging  between  48%  and  67%  (at  least  a  2:1  ratio).  With  the 
absence  of  physical  failure  analysis  reporting  data,  which  does  not  exist  in  field 
maintenance  data,  it  is  extremely  difficult  to  attempt  to  quantify  which  defect  groups 
might  be  classified  as  latent  (workmanship)  or  inherent  (design).  Intuitively,  Type 
1  repair  without  parts.  Type  6,  and  Type  2  conditions  would  more  than  j£rely 
contain  the  bulk  of  latent  related  defects;  and  the  Type  1  repair  with  parts  would 
contain  mainly  the  inherent  or  design  sensitive  defects.  It  should  be  noted  that 
Type  2  defects  are  almost  non-existent  in  any  of  the  maintenance  reporting  systems. 
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Figure  5l  Annual  historical  trends  1975  —  1985  (Navy  3M  database). 
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NON  ESS 

Figure  6.  ESS  effectivenen  assessment. 


In  assessing  the  effect  of  ESS  on  each  equipment  during  growth  evolution,  Tr 
bles  4  and  5  were  developed  to  contrast  the  growth  of  the  non-ESS  equipment  over 
the  11  year  span.  A  comparison  of  the  rate  distributions  for  the  first  two  years  of 
field  operation  was  made  with  the  latest  two  years  to  determine  the  effect  (if  any)  of 
engineering  changes  and  system  improvements  on  the  latest  configured  non-ESS 
groups.  The  comparisons  are  made  for  the  Type  l  repairs  with  parts  (to  reflect  po¬ 
tential  inherent  changes)  and  the  combined  Type  1  repairs  without  parts,  plus  Type 
6  and  Type  2  actions  (to  reflect  potential  latent  rate  changes).  The  results  as 
shown  in  Table  4  indicate  that  the  most  'ignificant  improvement  of  the  non-ESS  pop¬ 
ulation  over  the  years  is  contributed  by  the  inherent  type  rate  (Type  1  repair  with 
parts) .  This  would  be  expected  as  a  result  of  the  design  and  reliability  improve¬ 
ment  change^  incurred  over  the  years.  The  latent  defect  type  group,  on  the  other 
hand,  is  for  the  most  part  significantly  deteriorating  (negative  improvement  %),  in¬ 
dicating  that  any  improvement  activities  did  not  affect  workmanship  and  related  main¬ 
tenance  induced  type  actions. 

Table  5  contrasts  the  last  two  years  of  performance  of  the  non-ESS  population 
with  that  of  the  ESS  population  to  provide  a  transitional  growth  effect  in  going  from 
the  latest  field  improved  hardware  to  the  newly  delivered  ESS  hardware.  In  these 
comparisons,  although  the  'Type  1  repairs  with  parts  (inherent  related  category) 
rates  are  improved  across  the  board,  the  rate  reduction  is  not  as  great  as  the 
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TABLE  4.  Non-ESS  growth  affect*  «  »  function  of  type  action  distribution!. 
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TABLE  5.  Transitional  growth  &  effect  —  Non-ESS  »!  ESS  groups. 
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change  in  the  Type  1  repair  without  parts  plus  Type  8  and  Type  2  actions  (latent 
related  category)  which  provides  the  predominant  weight  on  the  overall  removal  rate 
reduction.  The  sensitivity  of  this  effect  is  predominantly  ESS  oriented. 

However,  since  inherent  and  latent  characteristics  are  extremely  difficult  to 
quantify  without  supporting  failure  analysis  data,  the  most  significant  attributes  that 
can  be  derived  as  a  result  of  effective  screening  are: 

•  Reduction  in  overall  removal  rate 

•  Stabilization  of  frequency  and  dispersion  patterns. 

Figure  7  shows  the  dispersion  of  each  LRU  population  removal  rate  on  a 
monthly  basis  for  a  five  year  period  (1981-1985)  in  order  to  assess  the  removal  fre¬ 
quency  patterns  of  the  ESS  hardware  with  the  comparable  level  non-ESS  growth 
hardware.  The  bell-shaped  frequency  curves  are  the  frequency  distribution  of  the 
monthly  averages,  which  were  graphed  to  determine  if  the  frequency  pattern  of  the 
non-ESS  vs  ESS  population  of  the  LRU  were  in  any  way  related,  since  they  are  es 
sentially  of  the  same  hardware  functionally.  Each  population  (ESS  vs  non-ESS)  had 
60  reporting  points  (five  years  x  12  months/year).  The  dispersion,  in  a  qualitative 
sense,  is  expressed  by  the  broadness  cf  the  range  values  about  the  mean  or  average 
rate  for  the  population.  Equipment  A,  as  an  example,  had  monthly  rates  for  the 
non-ESS  population  that  ranged  between  10  and  50  actions  per  1000  flight  hours, 
while  the  ESS  group  ranged  between  five  and  25.  The  percent  frequency  on  the  Y 
axis  is  the  percentage  cf  time  the  value  appears  over  the  five  year  span.  The  more 
consistent  the  removal  rate,  the  higher  the  frequency  should  be  about  the  average 
value.  With  respect  to  Equipment  A,  non-ESS,  the  approximate  frequency  about  the 
average  (31.3)  is  only  25°,  while  that  of  the  ESS  population  (14.3)  is  near  60%. 

The  cumulative  frequency  curves  represent  the  percentage  of  values  that  fall 
below  a  certain  value.  For  Equipment  A,  approximately  50%  of  the  monthly  values 
were  30  or  less  for  the  non-ESS  population,  while  the  ESS  population  had  100%  of  its 
values  less  than  30.  In  contrast,  the  ESS  population  had  90%  of  its  values  at  20  or 
less,  while  the  non-ESS  population  had  only  20%  at  20  or  less. 

The  frequency  distributions  in  every  case  history  comparison,  ESS  vs  non-ESS, 
barely  overlap  and  in  about  every  case  the  average  of  the  non-ESS  population  does 
not  appear  at  all  in  the  ESS  distribution.  No  further  statistical  manipulation  is 
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Figure  7.  Frequency  distribution  of  removal  rates  ESS  vs  non- ESS  equipment  groups. 


needed  to  conclude  that  these  populations  of  like  hardware  are  not  related  on  this 
basis  and  that  the  ESS  populations  are  significantly  more  stable  about  the  average 
values. 

With  the  common  ground  of  the  ESS  populations  being  that  they  were  all 
screened  as  new,  the  results  tend  to  indicate  that,  as  a  result  of  ESS  testing  on  at 
least  new  production  hardware,  the  overall  removal  rates  and  dispersions  over  a  sig¬ 
nificant  period  of  time  have  been  reduced  and  stabilized  (less  dispersion  about  the 
average).  The  characteristics  of  these  dispersions  in  terms  of  make-up  as  a  func¬ 
tion  of  the  specific  distribution  of  removals  per  unit,  and  any  skewness  effects 
derived  there  from,  are  discussed  in  Section  4. 
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4  -  SELECTION  CRITERIA  ANALYSIS 


In  the  field  implementation  of  ESS,  three  major  areas  must  be  addressed  before 
equipment  selection  decisions  can  proceed.  These  are: 

•  ESS  Sensitivity  Potential  -  equipment  maintenance  rate  history  sensitivity  to 
ESS  and  the  criteria  basis  to  support  the  decisions 

•  Level  of  Assembly  Testing  -  equipment  population  and  level  of  assembly  sen¬ 
sitivity  to  defect  distribution  and  repairability  factors 

•  Equipment  Age  -  equipment  age  in  terms  of  years  of  service,  and  the  poten¬ 
tial  effect  of  ESS  in  both  the  near  term  (i.e.,  can  the  equipment  survive?) 
and  the  long  term  (i.e.,  is  there  a  payoff  in  extended  reliability  and  ser¬ 
vice?)  . 

In  the  management  of  these  processes,  initial  decisions  as  to  the  selection  of 
equipment  to  test,  at  least  on  a  noneconomic  basis,  is  possible. 

4.1  ESS  SENSITIVITY  POTENTIAL 

It  should  be  understood  that  quality  of  testing,  not  quantity,  is  the  objective. 
Further,  the  objective  is  not  to  attempt  elimination  of  every  conceivable  defect,  since 
this  is  physically  and  economically  impossible.  Rather,  the  objective  is  to  dilute  the 
defective  population  as  optimally  and  as  quickly  as  possible  within  cost  constraints 
without  affecting  the  standing  readiness  of  operational  weapon  systems  and  field  ac¬ 
tivities.  Thus  the  approach  for  equipment  selection  is  to  establish: 

•  High  removals  for  cause 

•  Highest  potential  for  reliability  improvement 

•  High  bad  actor  sensitivity. 

4.1.1  Removals  For  Cause 

A  review  of  the  removal  history  of  avionics  of  various  aircraft  weapon  systems 
indicates  that  the  removals  are  concentrated  in  relatively  few  equipments.  As  shown 
in  Fig.  8,  approximately  10%  of  the  avionic  equipment  was  installed;  as  identified  by 
Work  Unit  Codes  (WUCs)  5  (Instrumental  Navigation),  6  (Communications/Naviga- 
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Figure  8.  Composite  distribution  of  %  of  avionic  removals  vs  %  of  electronic/avionic  enuipment  contribution. 


tion),  and  7  (Weapon  Systems);  in  the  F-14A,  A-6E,  E-2C,  F-18A  and  EF-111A 
aircraft  accounted  for  90%  of  the  LRU  removal  actions  on  those  aircraft.  This 
suggests  that  candidate  equipment  for  ESS  testing  come  from  this  population  of  high 
removal  units  since  they  have  the  highest  payoff  potential. 

Taking  this  one  step  further,  ranking  the  avionic  equipment  and  LRUs  by  high 
removals  for  each  of  the  aircraft  weapon  systems  provides  the  average  cumulative 
rank  distributions  as  shown  in  Fig.  9.  The  equipment  rank  (3  digit  WUC)  provides 
between  80  to  90%  of  the  removals  in  the  electrical  avionic  categories  (WUC,  5,  6  and 
7)  within  the  top  ten  ranked.  A  top-down  ranking  of  the  LRUs  (4  digit  WUC)  pro¬ 
vides  between  60  to  70%  within  the  top  15  to  25  ranked  LRUs.  Selection  judgement 
should  be  used  in  qualifying  the  high  contributor  LRUs.  Factors  to  be  considered 
include: 

•  Hardware  serialization 

•  Repairable  status  (it  is  desirable  to  select  repairables  only,  not  expendables 
or  disposables) 

•  Clear  equipment  identification  and/or  part  number  (e.g. ,  AN/ALQ-XXX) 

•  No  miscellaneous  (catchall),  or  Not  Otherwise  Coded  (NOC)  categories. 
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Figure  9.  Average  top  down  rank  distribution  of  electronic/xionic  tyytem/LRUt 
iWUC  categories  S,  6,  &  7), 

The  same  conclusions  apply  to  ground  electronics  since  similar  patterns  of  concen¬ 
tration  have  been  observed  for  electronic  type  hardware. 


4.1.2  Potential  For  Reliability  Improvement 

The  basis  for  high  potential  reliability  improvement  is  the  relationship  between 
the  measured  Mean  Time  Between  Removals  (MTBR)  and  the  specified  Mean  Time  Be¬ 
tween  Failure  (MTBF)  and  expressed  as  the  discrimination  ratio  (MTBF/MTBR).  The 
specified  MTBF  in  this  sense  is  the  design  or  operational  reliability  measure,  as  de¬ 
fined  by  handbook  predicting  techniques  (e.g.,  MIL-HDBK-217),  specified  contrac¬ 
tual  goals  (e.g.,  warranties),  field  operational  objectives  (e.g.,  R5lM  2000  targets), 
or  logistics  planning  goals  (e.g.,  wartime  loading  levels).  This  relationship  between 
specified  and  actuals  is  typically  illustrated  in  Fig.  10  ant-'  shows  that  Discrimination 
Ratios  (DR)  of  greater  than  one  would  provide  the  best  potential  for  implied  relia¬ 
bility  improvement  (in  the  sense  frequency  reduction).  Therefore: 


MTBF 

MTBR 


>  1 ;  has  the  highest  reliability  impact  potential  and 
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Figure  10.  Difference  effects  between  measured  removals  &  jpecrf'ad  reliability. 

£  1 ;  has  the  least  reliability  impact  potential 

MTBR 

It  would  seem  that  the  larger  the  ratio  (greater  than  one),  the  more  effect  ESS  will 
have  on  the  item's  potential  reliability  improvement.  Table  6  provides  the  DBRs  for 
the  case  history  LRUs  for  noth  ESS  and  non-ESS  populations.  In  each  case  the  ini¬ 
tial  ratio  of  specified  to  non-ESS  removals  is  greater  than  one,  Ls  the  range  of  2.4 
to  32.6.  Overall,  the  average  ratio  improvement  is  2:1,  Irrespective  of  the  initial 
ratio  magnitude  greater  than  one.  It  is  obvious  from  the  results  that  no  clear  cor¬ 
relation  can  be  drawn  '  ith  respect  to  specified  or  predicted  levels,  and  the  actual 
rates.  This  implies  that  the  defects  removed  are  independent  of  the  design  or  pre¬ 
dicted  failure  rate;  only  the  quality  attributes  are  affected  by  ESS. 

TABLE  6.  Impact  of  ESS  on  reliability  improvement  potential. 


EQUIPMENT 

NO.  UNITS 
PcR  SYSTFM 

SPECIFIED 

MTBF 

NON-ESS 

MTBR 

ESS 

MTBR 

|  DISCRIMINATION  RATIO  | 

%  RATIO 
REDUCTION 

NON-ESS 

ESS 

A 

1 

420 

32 

70 

13.1 

6.0 

54 

B 

1 

1070 

66 

127 

31.4 

16.3 

48 

C 

1 

202 

S3 

149 

2.4 

1.4 

42 

0 

t 

3000 

92 

278 

32.S 

10.8 

67 

E 

6 

5000 

570 

1110 

8.8 

4.5 

49 
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To  illustrate  this  effect,  consider  two  equipments  (x)  and  (y).  Equipment  (x) 
has  a  specified  MTBF  of  100  hours  and  (y)  has  an  MTBF  of  1000  hours.  By  virtue 
of  the  manufacturer,  ten  workmanship  defects  are  inserted  into  each  equipment.  If 
it  is  assumed  that  during  the  normal  course  of  operation  of  the  equipments  (x)  and 
(y)  the  defects  are  precipitated  within  1000  hours  of  operation,  the  removal  distri¬ 
bution  and  resulting  discrimination  ratio  (DR)  for  each  equipment  would  be  as  follows: 

Rmvls  Due 


to  Failure 

Rmvls  Due 

Hrs 

Specified 

MTBF 

to  Workmanship 

Equip 

(T) 

MTBF 

T 

Defects 

X 

1000 

100 

10 

10 

y 

1000 

1000 

1 

10 

Total 

Meas.  MTBR 

DR 

Rmvls 

T 

MTBF 

Equip 

(R) 

R 

MTBR 

X 

20 

50 

2:1 

y 

11 

91 

11:1 

Each  equipment 

shows  the  same  thing  (there 

are  10 

workmanship  defects) ,  but 

equipment  x  appears  less  affected  on  the  surface  than  y  since  it  is  expected  to  have 

a  low  MTBF, 

and  equipment  (y)  is  probably  giving  the  best  indication  that  something 

is  wrong.  If 

we 

increase 

the  operating  time  to 

10,000  hours  (assuming  that  the  de- 

fects  will  be  precipitated  within  the  10,000  hours),  the 

removal  distribution  for  each 

equipment  would  now  look  as  follows: 

Rmvls  Due 

to  Failure 

Rmvls  Due 

Hrs 

Specified 

MTBF 

to  Workmanship 

Equip 

(T) 

MTBF 

T 

Defects 

X 

10,000 

100 

100 

10 

y 

10,000 

1000 

10 

10 

Total 

Meas.  MTBR 

DR 

Rmvls 

T 

MTBF 

Equip 

(R) 

R 

MTBR 

X 

110 

90 

1.1:1 

y 

20 

500 

2:1 

xxue 
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As  noted  in  the  ESS  discrimination  ratio  column,  the  DRs  in  each  case  still  ex¬ 
ceed  one.  Thus,  either  the  retention  of  residual  latent  defects  is  not  completely 
removed  by  the  screen,  which  is  realistically  possible  (since  the  efficiency  of  screens 
is  not  100°o),  or  the  true  MTBF  of  the  unit  does  not  meet  or  exceed  the  specified 
MTBF  which,  without  further  discussion  of  failure  definition,  is  probable  for 
significantly  higher  DRs.  In  either  case,  you  would  not  continue  or  re-test  units 
previously  tested. 

It  is  possible  to  have  DRs  less  than  one  (where  the  measured  MTBR  meets  or 
exceeds  specified  MTBF)  as  well  as  workmanship  defects,  since  defects  do  not  dis¬ 
criminate  as  a  function  of  the  unit's  reliability.  However,  it  is  not  likely  that  these 
defects  will  affect  the  intended  design  performance  of  the  unit  or  system,  and  they 
would  not  be  considered  candidates. 

Therefore  the  selection  attributes  for  field  reliability  improvement  potential  via 
ESS  should  consider: 

•  The  units  contribution  to  the  total  weapon  system  removal  rate;  the  higher 
the  contribution,  the  more  the  potential 

•  The  number  and  configuration  of  units  required  per  system  to  affect  the  ag¬ 
gregate  rate 

•  The  actual  rate  falls  below  expected  or  specified  irrespective  of  degree. 

4.1.3  Bad  Actors  Sensitivity 

On  the  basis  of  average  rates,  as  described  in  the  previous  section,  each  unit 
in  the  population  would  have  to  be  tested  to  achieve  an  aggregate  effect.  However, 
this  is  not  the  case  in  a  non -homogeneous  process  where  not  all  units  are  defective 
in  a  quality  sense.  Further,  each  equipment  in  the  field  population  has  an  estab¬ 
lished  history  or  field  process  average,  which  provides  the  basis  for  selection. 
Table  7  and  Figure  12  show  how  removals  are  distributed  by  serial  number.  Again 
concentration  effect  is  evident  since  75%  of  the  population  removals  are  caused  by 
50%  of  the  serial  numbers.  In  addition,  the  worst  20%  of  the  serial  numbers 
contribute  up  to  45%  of  the  equipment  removals.  This  would  be  typical  of  a  quality 
process  effect  showing  up  the  potential  bad  actor  boxes  by  their  higher  than  av¬ 
erage  removal  frequency  thus  affecting  the  population  average.  This  would  imply 
that  selecting  bad  actors  (or  loser  boxes)  by  serial  number  from  a  population  of 
serialized  LRUs,  provides  a  means  for  improving  the  total  aggregate  removal  rate  by 
screening  only  a  select  number  of  units. 
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TABLE  7.  Cumulative  %  rank  iwM  number  removal  dirtribution  for  n on-ESS  equipment  population*. 


|  CUMULATIVE  %  RANK 

REMAINING 

SERVIC 

EQUIPMENT 

10X 

20% 

30% 

40% 

|  50% 

>  50% 

TOTALS 

YRS 

A  N 

111 

222 

333 

444 

551 

1106 

11 

RMVLS 

2690 

5029 

6783 

8186 

9356 

2339 

11695 

%  RMVLS 

23 

43 

58 

70 

80 

20 

B  N 

97 

194 

291 

388 

485 

485 

970 

11 

RMVLS 

1689 

2703 

3543 

4167 

4697 

1464 

5631 

% RMVLS 

30 

48 

63 

74 

82 

18 

C  N 

85 

170 

255 

340 

425 

424 

849 

11 

RMVLS 

1205 

1961 

2494 

2983 

3257 

1188 

4445 

%  RMVLS 

27 

44 

56 

67 

73 

27 

D  N 

103 

206 

309 

412 

515 

514 

1029 

11 

RMVLS 

1060 

1656 

2221 

2625 

2908 

1130 

%  RMVLS 

26 

41 

55 

66 

72 

28 

E  N 

38 

76 

114 

152 

190 

187 

377 

6 

RMVLS 

600 

861 

r>\r 

5JU 

1064 

1129 

201 

1330 

%  RMVLS 

45 

65 

75 

80 

85 

15 

WT  AVG. 

%  RMVLS 

27 

45 

59 

70 

78 

22 

N  -  NO.  OF  DIFFERENT  LRU  SERIAL  NUMBERS 

DATA  SOURCE  3M  DATA  1975-1985 
R87-3772-013(T) 


%  OF  RANKED  SERIAL  NUMBERED  LRU*  REP0R1 ING 
Ra7-3772-014(T) 

Figure  12.  Avenge  distribution  of  removal*  by  trial 
number*  reporting. 


In  establishing  a  bad  actors  program,  the  objective  is  to  minimize  the  number  of 
units  that  should  be  tested  and  to  identify  only  those  units  which  will  provide  the 
most  potential  benefit  from  an  ESS  effectiveness  point  of  view. 

To  illustrate  this  aspect,  consider  an  equipment  with  an  MTBF  of  100  hours, 
with  a  defective  rate  of  two  defects  per  unit,  with  100  units  in  the  field.  If  as  be¬ 
fore  in  the  illustration  of  Subsection  4.1.2,  the  defects  were  to  precipitate  out  within 
1000  hours,  at  the  end  of  1000  operating  hours  per  unit  each  unit  would  in  effect 
have  12  removals.  The  process  average  p  would  then  be: 

Total  Removals  R  1200 
=  -  =  —  =  - =  12 

Total  Units  N  100 

If  the  specified  tli'BF  of  100  hours  were  achieved,  then  the  expected  process  aver¬ 
age  (ve)  for  1000  hours  would  be: 

T  _  1000  _ 
e  MTBF  100 

The  comparison  between  the  actual  and  expected  would  conclude  that  all  units  were 
defective,  and  that  the  actual  exceeds  the  expected: 
u  i  ve 

If  however,  the  defectives  were  contained  in  only  25%  of  the  units  (25  units),  then 
the  frequency  of  removals  would  take  on  a  distribution  as  shown  in  the  table  below: 


N 

R 

V  =  R/N 

No.  Units 

No.  Removals 

Process  Avg 

25 

450 

18 

75 

750 

10 

100 

1200 

12 

The  overall  process  average  is  still  12,  but  the  defectives  are  isolated  to  only  25%  of 
the  population.  This  results  in  a  typical  frequency  distribution  as  shown  in  Fig. 
13. 

The  bad  actor  population  will  tend  to  have  removals  significantly  higher  than  the 
process  average.  The  removals  for  the  individual  unit  criteria  would  then  be: 
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POPULATION 
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REMOVALS/UNIT 
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Figura  13.  Typical  bad  actor  distribution  affect 


D  >  y  =  bad  actor  frequency 
where:  D  is  the  next  largest  integer. 

Any  unit  having  D  removals  or  more  would  be  a  candidate  for  selection.  The  un 
in  the  example  therefore  having  13  removals  or  more  would  be  selected  and  wot 
effectively  result  in  picking  all  25  defective  units. 


4. 1.3.1  Bad  Actors  Criteria  -  In  selecting  bad  actors,  the  LRU  serializatior 
reporting  in  the  field  maintenance  system  (AF/D056)  provides  the  means  to  ident 
and  determine  the  distribution  of  removals  by  LRU  serial  number.  Figure  14  pi 


vides  the  distribution  of  removals  per  unit  for  each  of  the  five  non-ESS  case  hist 
populations  for  the  last  two  years  (1984-1985)  of  operation.  In  all  cases,  tht 
distributions  are  skewed  with  75%  or  better  of  the  units  having  the  average  numt 
of  removals  (y)  or  less  and  the  remaining  25%  providing  the  significantly  more  th 
the  average  number  of  removals.  The  table  below  summarizes  the  results  giving  t 
(y)  and  (D)  values,  and  the  percentage  of  each  of  the  units  reporting  populatio 
that  would  be  selected  for  ESS  testing. 


%Qf  UNITS  REMOVED 


R§7-3772-054(T) 


EQUIPMENT  e 

Figur*  14.  DHtribution  of  rwwwl»  P »  »»«<• 
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No. "Bad  Actors" 


No.  Units 

Total 

No.  Units 

Equipment 

Reporting 

Removals 

y 

D 

D  >  y  % 

Units  Reporting 

A 

390 

821 

2.1 

3 

101 

26% 

B 

243 

535 

2.2 

3 

61 

25% 

C 

345 

518 

1.5 

2 

104 

30% 

D 

230 

345 

1.5 

2 

68 

30% 

E 

109 

153 

1.4 

2 

26 

24% 

4. 1.3.2  Bad  Actor  Selection  Effectiveness  -  Figure  15  illustrates  the  improvement 
removal  rate  and  removals/unit  for  the  non-ESS  populations  if  the  bad  actors  were 
removed  for  screening.  This  is  demonstrated  by  ranking  the  serial  numbered  units 
by  removals  for  the  1984-1985  reporting  period.  The  numbers  reflected  by  the  first 
column  of  the  bar  graph  coincide  with  the  statistics  provided  in  the  table  of  Sub¬ 
section  4. 1.3.1.  The  ESS  level  process  average  (y)  is  based  on  the  ESS  population 
rate  for  the  same  period.  This  essentially  says  that  the  objective  is  to  pull  and 
screen  the  non-ESS  bad  actors  in  order  to  reduce  the  non-ESS  population  process 
average  to  as  close  to  the  ESS  level  as  possible.  The  Bad  Actor  %  rank  reduction 
on  the  X  axis  is  the  rate  at  which  the  ranked  serial  numbers  are  being  pulled. 
Therefore,  if  a  population  consisted  of  100  units,  5%  would  result  in  the  first  5 
ranked  serial  numbers  by  removals  being  pulled,  25%  would  be  the  top  25  ranked, 
etc. 


To  reflect  the  impact  of  what  the  returned  field  ESS  units  might  have  on  a  real 
world  scenario,  an  equivalent  number  of  units  were  selected  from  the  ESS  population 
and  added  to  the  non-ESS  population.  Therefore,  the  subsequent  columns  are  the 
rate  at  which  the  non-ESS  units  are  pulled  and  replaced  by  effective  ESS  units  until 
a  normalized  level  was  achieved. 

As  noted  from  the  Y  axes,  the  removal  rate  (MA/1000  FH)  and  process  average 
(u)  (removals/unit)  of  the  non-ESS  population  significantly  reduce  to  a  normalized 
level  resulting  in  between  74  to  97%  gain  in  the  process  average  within  30  to  40%  of 
the  high  ranked  units. 

The  identifiable  "D  Limit”  noted  on  each  chart  is  the  point  in  the  rank  dis¬ 
tribution  where  the  bad  actor  criteria  is  no  longer  exceeded  (e.g. ,  for  equipment  A, 


"D"  was  3,  at  the  "D  limit”  identified  for  equipment  A  is  where  the  serial  numbered 
units  have  two  removils/unit  or  less) .  This  occurs  within  25%  of  the  total  serial 
number  reporting  and  results  in  a  93%  gain.  This  means  that  by  pulling  the  bad  ac¬ 
tors  in  accordance  with  ihe  selection  criteria  defined,  the  total  non-ESS  population 
process  average  is  reduced  to  within  7%  of  the  ESS  population  process  average.  The 
gain  distribution  in  terms  of  Type  1  repairs  with  parts.  Type  1  repairs  without 
parts  and  Type  6  and  Type  2  actions  at  the  "D  limit"  is  provided  in  the  table  for 
each  equipment  bar  graph.  The  shading  of  the  bars  indicate  the  contribution  of 
each  type  of  action. 

In  effect,  a  10  year  old  hardware  is  actually  performing  as  well  as  the  newer 
screened  hardware.  This  implies  that  not  only  are  the  highest  percentage  of  re¬ 
movals  attacked  via  the  bad  actors,  but  the  population  of  remaining  good  hardware 
approaches  that  of  the  new  ESS  population  operational  levels  with  the  average  gain 
in  removal  rate  on  the  order  of  between  63%  and  93%  with  only  between  24%  and  30% 
of  the  bad  actor  units  removed  and  theoretically  screened  or  otherwise  replaced  by 
screened  units.  Further,  as  noted  by  the  gain  distributions,  the  Type  6  actions  are 
almost  gaining  by  a  margin  of  2:1  over  either  Type  1  actions  (with  parts  or  without 
parts).  The  bad  actor  distributions  have  higher  than  normal  false  alarms  and  cannot 
duplicate  conditions;  this  is  highly  conducive  to  ESS  sensitive  quality  defects. 

These  observations  on  LRU  bad  actors  conclude  that  this  is  an  issue  that  war¬ 
rants  strong  consideration  and  further  investigation,  since  they  definitely  identify 
with  high  quality  defect  cases.  Investigations  at  the  SRU  level  cannot  be  accom¬ 
plished,  since  SRU  serialization  is  inconsistent  and  not  separated  to  any  degree  in 
the  maintenance  reporting  systems. 

4.2  LEVEL  OF  ASSEMBLY  TESTING 

In  dealing  with  level  of  assembly  testing,  it  must  be  first  understood  that  in  a 
field  scenario  all  process  control  is  lost.  Lot  homogeneity,  component  and  lower  lev¬ 
el  of  assembly  in  process  controls  do  not  exist,  and  any  resemblance  to  these  may 
have  been  long  since  lost  due  to  interchangeability  and  configuration  changes.  Fur¬ 
ther,  what  is  being  dealt  with  is  a  top-down  philosophy;  the  field  unit  is  fully  as¬ 
sembled  and  not  sensitive  to  in-process  control  because  the  lower  level  population 
cannot  be  eliminated  all  at  once,  unless  the  unit  is  effectively  overhauled  with  lower 
level  screened  assemblies  and  parvs.  Conversely,  an  LRU  screen  subjects  the  total 


population  of  lower  level  assemblies  to  stress  all  at  once.  On  a  per  repair  basis,  the 
effect  of  lower  level  ESS  is  virtually  non-existent  since  only  one  defect  can  be 
eliminated  at  a  time,  and  it  becomes  highly  sensitive  to: 

•  The  true  number  of  defectives  that  exist  in  the  unit,  which  is  not  kr.jwn 

•  The  number  of  times  the  higher  level  of  assembly  is  repaired  in  its  lifetime. 

The  question  that  arises  is  whether  or  not  lower  levels  of  field  screening  can 
be  justified  not  only  on  the  basic  of  test  effectiveness,  but  also  as  to  their  practi¬ 
cality  from  the  point  of  view  of  defective  reduction  at  the  higher  levels  of  assembly. 

In  none  of  the  five  case  histories  studied  were  parts  or  lower  levels  of  assembly 
screened  (other  than  as  required  by  the  standard  part  specification),  nor  were  any 
of  the  logistic  spare  parts  and  assemblies  that  used  in  the  field  repair  process  of 
these  equipment  screened.  As  noted  in  the  ESS  data  history  profiles  for  the  equip¬ 
ments,  in  Section  3,  there  has  not  been  any  degradation  nor  has  it  had  any  bearing 
on  the  removal  rates  over  the  last  four  to  five  years. 

Table  8  provides  the  total  number  of  SRU  and  lower  level  assemblies  included  in 
the  LRU  population  for  both  non-ESS  and  ESS  groups.  The  corresponding  removal 
rates  per  component  (X)  and  equivalent  MFHBR  is  provided.  Based  on  the  data  of 
Table  8,  Table  9  summarizes  the  effect  that  would  occur  if:  (1)  the  component  of  a 
non-ESS  LRU  was  replaced  or  repaired  by  a  lower  level  ESS  component,  and  (2)  an 
ESS  LRU  was  repaired  by  installing  a  non-ESS  component.  These  effects  are  ex¬ 
pressed  as  the  percentage  of  improvement  or  degradation  in  removal  rate.  The  cor¬ 
responding  average  removal  per  LRU  percentage  is  obtained  by: 

%  Improvement  = 

XLRUN  N  =  Non-ESS 

E  =  ESS 

%  Degradation  =  *CN^_\CE  C  =  Component 

XLRUg 

As  noted  by  the  defective  rates  per  device  (Table  8)  and  their  effects  (Table 
9),  the  defective  contribution  becomes  more  and  more  complex  as  the  level  of  assem¬ 
bly  approaches  the  piece  part.  If,  as  noted  the  average  rate  of  repair  of  a  non-ESS 
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TABLE  8.  Lower  level  of  auembiy  removal  rate  distributions. 


NON- ESS 


EQUIP 

REPORTING 

LRU* 

TOTAL 

SRUs 

TOTAL 

HYBRIO/IC 

TOTAL 

OTHER  PARTS 

AVG 

LRU  RMVLS 
PER  VR 

AVG 

UTIL/LnU 

FH/YR 

A  N 

1106 

54194 

1.1  X  10® 

1.L  X  1 0® 

0.8 

30 

X 

.0313 

5  X  10-4 

2.5  X  10-5 

19X10-5 

MFHBR 

32 

2000 

40,000 

53,000 

B  N 

970 

20370 

2.8  X  105 

2.3  X  10® 

0.4 

35 

X 

.0152 

5  X  1C4 

3.6  X  IO-5 

4.4X10® 

MFHBR 

66 

2000 

28000 

225000 

C  N 

849 

43299 

2.3  X  10® 

29X10® 

09 

1  40 

X 

.0120 

1.7  X  IO*4 

3.2  X  10-« 

3.3  X  10-8 

1 

MFHBR 

83 

5800 

308000 

301000 

0  N 

1029 

15435 

1.7  X  105 

1.3  X  10® 

0.3 

33 

X 

.0109 

5  X  IO-4 

4.5  X  10-5 

6.2  X  10-® 

MFHBR 

92 

2000 

22000 

161000 

E  N 

377 

3016 

5.2  X104 

5.1  X  104 

0.5 

56 

X 

.0018 

1.8  X  10-* 

1  X  10-5  i 

1  X  IO*6 

MFHBR 

670 

5532 

100,000 

100,000 

ESS 

AVG 

AVG 

REPORTING 

TOTAL 

TOTAL 

TOTAL 

LRU  RMVLS 

‘  1TIL/LRU 

EQUIP 

LRUs 

SRUs 

HYBRID/IC 

OTHER  PARTS 

PER  VR 

FH/YR 

A  N 

587 

28763 

5.8  X  105 

78  X  10® 

0.6 

51 

X 

.0143 

2.2  X  IO*4 

1  X  10-5 

8  X  10"® 

MFHBR 

70 

4600 

1  JO,  000 

125,000 

B  N 

453 

9513 

1.3  X  105 

1.1  X  10® 

0.4 

66 

X 

.0079 

2.8  X  10-4 

2  X  10*5 

2.5  X  10-® 

MFHBR 

127 

3600 

50,000 

400.000 

C  N 

432 

22032 

1 .2  X  106 

1.2  X  10® 

0.4 

7.) 

X 

.0067 

1  x  io-4 

1.9X10-® 

1.9  X  10-® 

MFHBR 

149 

10,000 

520,000 

520,000 

D  N 

285 

4275 

4.8  X104 

3.6  X  10s 

0.3 

106 

X 

.0036 

1.8  X  IO"4 

1.6  X  IO'5 

2.1  X  10-® 

MFHBR 

278 

5500 

62000 

460.000 

E  N 

158 

1264 

2.2  X  104 

I  2  X  104 

0.3 

70 

X 

.0009 

1.1  X  IC5 

5.4  X  IO'® 

5.4  X  IO*® 

MFHBR 

1111 

1100 

185.000 

185,000 

X  -  REMOVAL  RATE  PER  FLIGHT  HOUR 

MFHBR  -  MEAN  FLIGHT  HOUR  BETWEEN  REMOVALS 


R  87-3  7  72-01 6  (T) 


4-18 


TABLE  9.  Improvement  n  degradation  of  LRU  effect*  « a  function  of  tingl e 


repair!  with  lower  lay  el  assemblies. 

*  IMPROVEMENT/REPAIR 
NON-ESS  LRU  REPAIR 

BY  ESS  ASSYt 

%  DEQRAOATI  ON/REPAIR 
ESS-LRU  REPAIR 

BY  NON-ESS  ASSYt 

EQUIP 

SRU 

HYB/IC 

OTHER 

PARTS 

SRU 

HYB/IC 

OTHER 

PARTS 

A 

0.9 

0.05 

0.02 

2 

0.1 

0.07 

8 

1.5 

0.15 

0.02 

2 

0.2 

os 

C 

0.6 

0.01 

0.01 

0.9 

0.6 

0.6 

O 

3 

os 

004 

6.7 

1.0 

1.0 

E 

a3 

0.4 

0.4 

10.5 

0.5 

os 
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LRU  is  in  the  range  of  0.3  to  0.8  per  year,  and  the  percentage  of  improvement  pos¬ 
sible  by  a  screened  SRU  is  between  0.6%  and  8.3%,  it  can  be  seen  that  it  would  take 
anywhere  from  10  to  100  years  to  double  the  reliability  of  the  LRU  via  repair  by 
screened  SRUs  (assuming  no  further  contamination).  As  the  level  of  assembly  gets 
lower  this  becomes  even  more  compounded. 

Conversely,  if  there  are  LRUs  that  have  been  screened  and  are  being  repaired 
by  lower  levels  of  assembly  that  have  not  been  screened,  the  rate  of  decay  is  simi¬ 
larly  slow  since  the  rate  of  LRUs  being  repaired  is  lower.  For  this  reason,  the 
screened  LRU  population  has  shown  no  degradation  over  the  last  five  years  of  field 
operation,  as  previously  noted.  This  is  without  discounting  state  of  the  art  im¬ 
provements  that  have  been  incorporated  in  the  parts  over  the  last  10  years. 

It  should  also  be  noted  that  for  those  cases  where  the  LRU  improvement  or 
degradation  is  high  per  removal  (e.g.,  equipment  E  with  8.3%  to  10.5%  rate  change 
per  removal)  this  occurs  in  the  less  complex  (e.g.,  lower  number  of  SRUs  per 
LRU),  higher  reliability  devices.  Therefore  a  scenario  of  high  density  SRU  removals 
per  LRU  should  be  given  consideration  as  a  potential  candidate  for  ESS  screening, 
especially  if  the  failure  rates  are  high.  Figure  16  provides  a  decision  making  aid  for 
Identifying  candidate  SRU  or  lower  levels  of  assembly  effects  on  the  LRU. 

The  abscissa  (X  axis)  is  the  ratio  of  total  removals  contributed  by  the  SRU  or 
lower  level  of  assembly.  The  right  ordinate  (Y  axis)  provides  the  potential  im¬ 
provement  achievable  of  the  SRU  or  lower  level  of  assembly  as  a  result  of  screening. 


A  100°o  improvement  is  a  virtual  elimination  of  the  component  as  a  removal  rate  con¬ 
tributor.  The  left  ordinate  of  the  graph  is  the  corresponding  improvement  that  is 
achievable  at  the  LRU  level  based  on  the  achieved  SRU  or  lo'-er  level  improvement. 
As  an  example,  if  an  SRU  or  lower'  level  of  assembly  contributes  50°o  of  the  total  LRU 
removals,  eliminating  it  completely  (or  100°..  improvement)  cannot  improve  the  LRU 
anymore  than  3C'.;,  since  you  cannot  gain  more  than  you  put  in.  Similarly,  if  the 
screen  is  only  30;  effective,  then  the  1  RV  improvement  can  only  be  25’.. 


For  these  reasons,  unless  it  can  be  justified,  based  on  the  graph  of  Fig.  16, 
the  rate  of  impact  at  a  lower  level  of  assembly  will  have  au  impact  at  the  LRU  level, 
and  there  is  no  gain  that  can  be  realized  by  .screening  the  lower  levels  of  assembly. 
If  there  are  relatively  !vw  SRi's  or  in  the  population  of  SRUs  one  which  provides  the 
predominant  distribution  of  removals,  then  there  is  a  potential  that  screening  the 
SRU  car,  achieve  the  vt  responding  removal  rate  improvement  at  the  LRU  level. 


RATIO  OF  SRU  REMOVALS  TO  LRU  REMOVALS 
SRU/LRU  % 
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Figure  16.  Improvement  in  LRU  removal  rate  as  a  function  of  SRU  removal  rate  contribution. 


.’hese  relationships,  however,  are  non-existent  in  the  field  maintenance  data 
cenarios  and  would  require  feedback  on  SRU  repairs  per  LRU  from  the  ALCs. 
udicious  screening  of  lower  level  (procured  for  logistic  purpose)  devices  while  at 
he  supplier  would  ensure  sound  quality  of  lower  levels  of  repair  elements  that  would 
>e  entering  "clean"  LRUs. 


1.3  EQUIPMENT  AGE 

Experience  with  equipment  age,  in  terms  of  years  of  service  and  potential  ef¬ 
fects  of  ESS,  does  not  exist  to  any  formal  degree  or  with  sufficient  background  to 
mpport  decisions  one  way  or  the  other.  Factors  affecting  equipment  age  include  as¬ 
pects  of  growth  as  a  result  of  years  of  reliability  improvement  and  upgrading,  as 
well  as  degradation  as  a  result  of  extended  use  and  potential  wearout.  These  fac¬ 
tors  must  be  counterbalanced  to  rationalize  the  potential  effect  from  ESS,  which  in 
one  case  can  significantly  improve  the  growth  characteristic  of  the  device  and  in  an¬ 
other  case,  degrade  it  in  that  it  can  potentially  be  destructive  to  the  equipment. 


From  the  detailed  removal  data  generated  for  the  five  equipments,  for  both 
non-ESS  and  ESS  populations  over  10  years,  growth  patterns  were  developed  using 
the  Duane  growth  model  (Ref  3)  as  described  by  the  exponential  growth  rate  equa¬ 
tion: 

ir. 


where  =  cumulative  removal  rate  over  (T)  years 

Xj  =  initial  removal  rcte  at  the  initial  year  (tp 
m  =  slope  or  growth  rate  parameter 


This  equation  plots  as  a  straight  line  on  log/log  coordinates. 

Table  10  provides  the  results  of  the  growth  analysis  and  shows  the  »*eiroval 
rates  and  corresponding  slopes  for  each  equipment  type. 

The  ESS  groups  exhibit  no  growth  in  removal  rate  over  the  four  to  five  years 
that  they  have  been  operational.  The  non-ESS  groups  have  some  degree  of  positive 
growth,  between  20%  and  2%,  over  the  10  years  of  performance. 
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TABLE  10.  Growth  trend  partmetm. 


EQUIP 

POPULATION 

INITIALRMVLS 

MA/1000FH 

*T 

CUM-RMVLS 

MA/1000FH 

M 

GROWTH 

RATE 

T 

VRS 

A 

NON  ESS 

40.7 

31.3 

0.2 

10 

ESS 

14.3 

14.3 

0 

4 

8 

NON  ESS 

15.6 

10.4 

0.17 

10 

ESS 

7.9 

7.9 

0 

4 

C 

NON  ESS 

10.0 

8.8 

0.05 

10 

ESS 

6.7 

6.7 

0 

4 

D 

NON  ESS 

8.2 

7.8 

0.02 

10 

ESS 

3.6 

3.6 

0 

4 

E 

NON  ESS 

11.6 

8.7 

0.16 

5 

ESS 

5.4 

5.4 

0 

5 
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Growth  projections  were  developed  as  shown  in  Fig.  17  (1)  and  (2)  (for  equip¬ 
ment  A)  to  establish  the  effect  that  ESS  would  have  on  field  performing  hardware 
that  was  developing  along  the  non- ESS  growth  line.  The  projected  field  ESS  line  is 
generated  by  projecting  back  the  ESS  level  to  what  it  might  have  been  if  it  had  been 
applied  to  the  non-ESS  population  10  years  ago.  At  the  point  where  the  projected 
field  ESS  line  intersects  the  normal  non-ESS  growth  line,  the  effect  of  ESS  would 
have  actually  no  growth  value. 

The  difference  between  the  non-ESS  removal  line  and  the  project  field  ESS  line 
at  any  time  (T  years)  provides  the  percentage  of  potential  improvement  in  the  re¬ 
moval  rate  that  could  be  realized  as  a  result  of  applying  ESS  at  that  point  in  the 
service  life  of  the  non-ESS  LRU.  Figure  17(3)  provides  the  “  improvement  growth 
pattern  lines  for  each  equipment  type.  Equipments  with  little  or  no  growth  effect 
essentially  conclude  no  change  over  time  (equipments  c  and  d).  The  percent 
improvement  points  are  then  plotted  on  the  log/log  coordinates  for  each  equipment 
type,  and  a  weighted  (by  removal  rate  density)  improvement  line  as  a  function  of 
age  is  developed.  This  line  (Fig.  17  (4))  represents  the  potential  improvements  in 
removal  rate  that  might  be  expected  as  a  function  of  the  service  life  of  the 
equipment.  The  results  tend  to  imply  that  hardware  with  greater  than  10  years  of 
field  service  would  offer  little  if  any  growth  potential  due  to  ESS. 
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FVgurt  17.  Equipment  aga  affact  extrapolation. 


4.4  SELECTION  CRITERIA  CONCLUSIONS 

Conclusions  developed  from  the  selection  criteria  assessment  have  identified 

that: 

•  High  removal  contributor  ranks  provide  the  prioritization  of  equipment  and 
LRUs  to  initiate  the  process  of  picking  candidate  hardware  for  screening. 
This  is  made  possible  by  the  fact  that  the  total  number  of  equipment  re¬ 
movals  from  a  weapon  system  is  nominally  contributed  by  a  relatively  small 
percentage  of  the  total  population  of  electronic  equipment  making  up  the  sys 
tern 

•  Bad  actor  selection  provides  a  process  for  selectirg  a  small  number  of  LRUs 
identifiable  by  serial  number,  from  the  high  contributor  population  which 
provide  the  highest  percentage  of  ESS  sensitive  defectives 


•  SRU  and  lower  level  component  screens  are  highly  sensitive  to  the  cont 
tion  the  level  hardware  is  actually  making  to  the  LRU  removal  ra 
Selection  at  lower  levels,  for  field  screening,  must  be  carefully  assess 
establish  that  the  scr.  en  will  have  an  impact  on  the  LRU  on  a  per  rt 
basis 

•  Comparisons  of  removal  rates  (MTBR)  to  predicted  or  specified  reliabi 
(MTBF)  provide  some  insight  to  potential  candidates  in  that  the  latent 
defects  will  suppress  the  true  failure  rate.  This  is  particularly  sign  if 
for  small  population  systems,  e.g.,  ground  radars,  ground  test  equipi 
etc.,  where  processes  such  as  bad  actor  selection  are  not  feasible.  Th 
crimination  ratio  of  MTBF ^  indicates  potential  candidate  selection.  The 

MTBR 

greater  the  magnitude,  the  greater  the  possibilities 

•  Equipment  age  and  growth  effects  cannot  be  clearly  quantified.  Extro 
tion  and  averaging  of  growth  experience  curves  of  the  five  case  histo 
tend  to  indicate  that  LRUs  tested  beyond  10  years  of  age  offer  litl 
improvement  benefit  as  a  result  of  screening.  This  cannot  be  supporte 
the  study's  case  history  ESS  population,  since  all  equipments  tested  v 
new  and  not  field  deployed.  The  nature  of  age  effects  can  only  1 
determined  as  a  function  of  experience  factors,  which  up  to  this  time 
not  been  available. 


5  -  ESS  TEST  PROFILES 


5.1  BACKGROUND 

In  1957  the  Advisory  Group  or*  Reliability  of  Electronic  Equipment  (AGREE), 
created  in  1952  by  the  Department  of  Defense  Research  and  Development  Board  to 
"monitor  and  stimulate  interest  in  reliability  and  recommend  measures  that  would  re¬ 
sult  in  more  reliable  equipment,"  published  its  recommendations.  These  included 
specific  requirements  for  establishing  environmental  test  profiles  to  be  used  during,' 
reliability  demonstration  testing.  It  was  also  suggested  that  these  same  conditions 
be  utilized  for  acceptance  testing  for  electronic  hardware.  Vibration  was  established 
as  one  of  ihe  environments  and  was  limited  to  a  sinusoidal  excitation  of  ±2g  at  a 
fixed  non-resonant  frequency  between  20  and  60  Hz.  This  form  of  vibration  per¬ 
sisted  for  years  and  was  used,  with  few  exceptions,  in  the  majority  of  electronics 
and  avionic  equipment  acceptance  tests  conducted. 


Evolving  from  the  McDonnell  Douglas  Mercury  and  Gemini  manned  spacecraft  pro¬ 
grams,  random  vibration  was  utilized  to  more  effectively  screen  workmanship  defects. 
The  unprecedented  success  of  the  Apollo  manned  space  program,  attributable  in 
large  measure  to  the  intensive  test  program  (Ref  4),  generated  some  new  thinking  in 
industry  and  the  military  concerning  the  utilization  of  effective  testing  (including 
random  vibration)  in  achieving  reliability  requirements.  Skeptics  still  maintained 
that,  while  those  techniques  might  work  for  Apollo  whose  vehicles  were  essentially 
"one  shot"  devices,  they  probably  would  not  be  effective  for  Imrdware  (such  as  air¬ 
craft  avionics)  which  had  to  survive  thousands  of  takeoff,  flight,  and  landing  hours. 
Grumman  decided  at  this  time  to  investigate  the  merits  cf  sine  and  random  vibration 
testing.  It  appeared  that  random  vibration,  which  provides  simultaneous  excitation 
of  many  modes  in  contrast  to  the  single  frequency  sine  test,  must  be  more  effective 
in  disclosing  manufacturing  defects. 


In  1972,  Grumman  embarked  on  an  investigation  to  determine  the  effects  of  en¬ 
vironmental  stimulation  of  workmanship  and  manufacturing  defects  typically  found  in 
avionic  equipment.  The  primary  objective  of  this  research  was  to  develop  a  test  em¬ 
bodying  those  environmental  screens  which  were  most  effective  in  detecting  latent 


mm 


defects  in  contemporary  hardware.  As  seen  in  Fig.  18,  a  major  conclusion  reachec 
in  an  earlier  Air  Force  study  (Ref  5)  conducted  by  Grumman  shows  that  vibratior 
and  temperature  are  responsible  for  the  majority  of  environmentally  related  field 
problems.  Experience  also  indicates  that  workmanship  defects  respond  to  these  sami 
environmental  stimuli  as  a  function  of  their  particular  sensitivity. 


salt,  :% 

ALTITUDE,  1  % 
SHOCK,  1% 


Figure  18.  Total  field  failure  distribution. 

5.2  RANDOM  VIBRATION  INVESTIGATIONS 

The  sparsity  of  random  vibration  application  data  prompted  Grumman  to  initiate 
a  laboratory  test  evaluation  structured  to  directly  compare  the  effectiveness  of 
sinusoidal  and  random  vibration  (Ref  6).  A  technical  approach  was  conceived 
wherein  the  time- to -failure  of  typically  occurring  defects  could  be  examined  under 
controlled  environmental  conditions  and  selected  durations.  Typical  workmanship  de¬ 
fects,  representing  80%  of  manufacturing  problems  found  in  avionic  hardware,  were 
selected  from  space  and  aircraft  test  and  field  failure  data.  These  defects  were  sim¬ 
ulated  in  quantities  considered  sufficient  for  analysis  and  were  inserted  Into  a  typi¬ 
cal  avionic  "black  box."  The  test  plan  pro/ided  for  a  total  of  100  simulated  defects 
to  be  included  in  any  given  test  matrix  of  different  levels  and  durations. 


Tests  were  conducted  using  sine  fixed  frequency,  sine  sweep,  and  random  vi¬ 
bration  excitations  at  different  levels  and  for  varying  periods  of  time.  Figure  19,  as 
an  example,  depicts  the  matrix  for  sine  sweep  testing.  Similar  matrices  were  devel¬ 
oped  for  sine  fixed  frequency  and  random  vibration  tests. 

TEST  SERIES  1  -  SINE  SWEEP 
5  -  5CO-  5Hz 


1  DURATION 

LOW- 
10  MIN 

MED- 
30  MIN 

HIGH- 
60  MIN 

LOW+1.59 

MED+5g 

HIGH+IOg 


EACH  TEST  - 100  FAULTS 
(20  OF  5  TYPES) 
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Figur«19.  Typical  test  matrix. 


The  results  clearly  indicate  that  random  vibration,  at  a  0.04g  /Hz  level  (Fig. 
20),  was  significantly  more  effective  than  either  of  the  sinusoidal  tests.  Figure  21 
compares  the  effectiveness  of  the  three  forms  of  vibration  for  two  of  the  most  com¬ 
mon  defect  types  at  levels  "typically"  used  in  acceptance  testing.  The  results  of 
this  comparison  are  obvious.  Figure  22  compares  the  "typical"  random  level  with  a 
t5g  level  for  each  of  the  sine  type  tests.  The  results  show  that  even  at  increased 
levels,  the  random  vibration  is  more  effective  (for  a  given  fault  type)  than  sine 
fixed  frequency  or  sine  sweep.  In  the  Fig.  23  comparison,  levels  of  vibration  up  to 


+3  dB/OCTAVE 


-3  dB/OCTAVE 


and  exceeding  qualification  were  used  for  the  sine  type  of  test.  Although  the  sine 
sweep  test  was  close  to  the  "typical"  random  test  or  both  failure  types,  it  required 
durations  of  approximately  one  hour  at  qualification  levels  (tlOg)  to  achieve  this 
type  of  effectiveness.  Testing  production  hardware  at  these  levels  and  durations 
would  certainly  present  a  potential  fatigue  problem  and  would  never  be  utilized  in  an 
acceptance  test.  The  "typical"  random  vibration  spectrum  achieved  its  maximum  ef¬ 
fectiveness  in  only  10  minutes  of  testing. 

2 

Some  concern  was  expressed  that  the  application  of  a  0.04g  /Hz  random  vibra¬ 
tion  level  would  cause  fatigue  and  structural  damage  if  applied  to  equipment  even  if 
that  equipment  had  proven  its  structural  integrity  during  qualification  tests.  During 
the  advanced  development  program  conducted  by  Grumman,  a  correctly  manufactured 
example  of  each  fault  type  was  inserted  in  the  test  article  as  a  control.  Even  after 
many  hours  of  exposure  at  the  Q.04g  /Hz  level,  not  one  of  these  correctly  manu¬ 
factured  examples  failed.  Further,  equivalency  analyses  performed  by  Grumman  and 
Wright- Patterson  Air  Force  Base  indicate  that  the  0.04g  /Hz  level  is  much  less  se¬ 
vere  than  qualification  levels  currently  used.  In  his  paper,  (Ref  7),  Dr.  Dreher 

2 

points  out  that  a  fatigue  test  level  of  Wp  =  0.10g  /Hz  is  equivalent  to  a  sinusoidal 

2 

level  of  only  G  =  ±2.5g.  He  further  indicates  that  it  takes  a  level  of  =  1.6g  /Hz 
to  be  equivalent  to  a  ±10g  sinusoid.  It  should  be  noted  that  these  equivalencies, 
developed  analytically,  apply  universally  to  any  type  of  equipment  undergoing  vibra¬ 
tory  excitation. 

Additionally,  Grumman  has  had  extensive  experience  in  the  use  of  random  vibra¬ 
tion  as  an  acceptance  test,  workmanship  screen  and/or  troubleshooting  aid.  During 
the  Lunar  Module  (LM)  space  program,  over  7,000  tests  were  performed.  In  all  the 
history  of  random  vibration  applied  at  the  0.040g^/Hz  level  at  Grumman,  no  known 
instance  of  degradation  or  subsequent  fieid  failure  attributable  to  the  vibration  test 
has  occurred. 

The  development  of  an  effective  random  vibration  screen  was  a  significant  mile¬ 
stone  in  the  evaluation  of  environmental  acceptance  testing.  A  review  of  flight 
hardware  application  data  (both  test  and  field)  indicated  that  exposure  to  the  random 
vibration  test  spectrum,  more  than  doubled  the  equipment's.  MTBF.  It  should  be 
noted  that  the  random  vibration  applied  was  only  10  minutes  in  duration  in  one  di- 


rection,  compared  to  many  hours  of  sine  vibration  formerly  used.  While  the  develop¬ 
ment  of  the  new  vibration  screen  was  proven  to  be  more  effective  and  shorter  than 
tests  previously  employed,  it  is  the  time  required  for  thermal  cycling  which  actually 
drives  the  test  costs. 

5.3  THFRMAL  CYCLING  INVESTIGATIONS 

Dun  t g  1976  the  program  evaluated  the  effects  of  various  thermal  cycling  rates 
on  defects  into  avionic  equipment  at  the  black  box  level.  The  purpose  of  this  eval¬ 
uation  was  to  establish  both  the  more  effective  cycle  and  the  optimum  exposure  dura¬ 
tion  required.  Data  could  be  obtained  on  the  slow  thermal  cycling  normally  used  for 
acceptance  of  LRUs,  but  similar  information  was  not  available  for  more  rapid  temp¬ 
erature  change  effects.  A  laboratory  program  was  therefore  developed  to  examine 
the  time -to -failure  of  typical  defects  under  controlled  thermal  conditions.  Seven  of 
the  most  common  generic  types  of  workmanship  faults  that  were  presumed  sensitive 
to  temperature  and  found  in  field  hardware  were  selected  and  stimulated  in  quantity. 
These  faults  were  inserted  into  a  typical  LRU  and  a  series  of  tests  was  conducted 
under  various  thermal  cycling  conditions. 

In  the  first  test  the  specimens  were  exposed  to  the  most  rapid  achievable  ther¬ 
mal  cycle  in  a  standard  tempera ture  chamber.  During  this  test  the  air  temperature 
was  varied  at  an  average  rate  of  approximately  5°C  (9°F)/min.  between  -54°C  (-65°F) 
and  +71°C  (+I60°F)  (Fig.  24).  Each  thermal  cycle  required  1.5  hours  to  complete. 


R67-3772*023fT)  TIME,  HRS 

Figur«24.  FUpid  thf  mil  cydt. 
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The  test  was  continued  to  a  total  of  168  hours  and  during  this  period  each  fault  was 
energized  and  monitored  on  a  full-time  basis.  Any  failures  occurring  during  the  ex¬ 
posure  period  would  be  instantly  recognized  by  the  monitoring  circuitry,  and  a  lamp 
lit  and  latched  for  either  intermittent  or  permanent  anomalies.  This  arrangement 
permitted  unattended  operation  (nights  and  weekends),  thereby  greatly  facilitating 
the  effort  and  permitting  maximum  use  of  available  calendar  time. 

Prior  to  initiating  the  second  test,  all  faults  were  refurbished  and  returned  to 
their  original  state.  The  second  test  was  then  conducted  by  exposing  the  speci¬ 
mens  to  a  thermal  cycle  which  was  the  standard  generally  now  used  by  industry  in 
acceptance  tests.  The  rate  employed  was  approximately  1.4°C  (2.5°F/min.  and  air 
temperature  was  varied  between  -54°C  (-65°F)  and  +71°C  (+160°F)  (Fig.  25).  Each 
cycle  required  six  hours  to  complete.  This  cycle  was  approximately  25po  of  the  rapid 
cycle  in  terms  of  cycles/unit  time.  Monitoring,  fault  detection  and  duration  of  test 
were  identical  to  those  employed  during  the  first  test. 


TIME  -  HOURS 
SLOW  THERMAL  CYCLE 

R87*3772-024(T) 


Figur#  25.  Thfrrul  cydt. 

Figure  26  presents  two  curves  which  summarize  the  test  results  for  the  fault 
types  and  compare  the  relative  effectiveness  of  the  two  thermal  cycles  imposed.  The 
rapid  cycle  detected  three  times  the  number  of  faults  disclosed  by  the  slow  cycle 
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Figurt  28.  Tit  rtsultt  —  riow  «  rapid  cydt. 


within  the  same  time  period.  Further,  the  point  of  diminishing  exposure  time  effi¬ 
ciency  was  40  hours  for  the  rapid  cyc’e  compared  to  approximately  50  hours  for  the 
slow  cycle.  Of  even  more  significance  is  the  fact  that  only  approximately  40  hours 
of  thermal  cycling  at  a  rapid  rate  are  required  to  remove  certain  latent  workmanship 
defects  compared  to  the  200  hour  average  duration  test  generally  employed  by 
industry.  Testing  beyond  the  40  hour  point  produces  little,  if  any,  additional 
screening  of  these  samples. 

5.4  OPTIMIZED  ESS  TEST  SEQUENCE 

Grumman  embarked  in  1977  on  an  investigation  to  measure  the  efficiency  of  ESS 
testing  if  one  were  to  combine  random  vibration  and  rapid  thermal  cycling  and  also 
optimize  the  sequence  of  its  application. 

Initially,  a  tradeoff  study  was  performed  to  investigate  sequential  versus  com¬ 
bined  environmental  exposure.  Combining  the  two  environments  did  not  offer  any 
technical  advantage,  i.e.,  no  synergism  was  apparent.  Furthermore,  the  cost  in¬ 
curred  by  dedicating  a  random  vibration  system  to  production  acceptance  testing 
clearly  prohibited  combining  the  environmental  exposures  for  acceptance  testing. 
The  study  therefore  concentrated  upon  a  sequential  schedule  of  rapid  thermal  cycle 
and  random  vibration  applied  at  the  LRU  level.  Additionally,  the  possible  advantage 
of  "prescreening"  complex/high-problem  PC3s  prior  to  their  installation  in  a  black 
box  was  very  attractive.  Again,  a  study  was  initiated  to  investigate  possible 


methods  of  applying  the  environments  and  monitoring  PCB  performance  without  the 
need  for  an  enclosure.  Although  applying  rapid  thermal  cycling  and  monitoring  the 
assembled  boards  seemed  easily  effected,  the  problems  associated  with  applying 
random  vibration  without  special  holding  fixtures  were  of  such  a  magnitude  as  to 
preclude  its  further  consideration.  It  was  therefore  decided  to  conduct  laboratory 
experiments  in  which  PCB  assemblies  were  exposed  to  rapid  thermal  cycling  only. 

As  in  previous  investigations,  the  technique  utilized  was  to  purposely  insert 
faults  into  the  test  equipment  and  then  determine  which  environmental  exposure  most 
effectively  stimulated  these  faults. 

It  should  be  noted  that  for  each  fault  inserted,  a  correctly  manufactured  exam¬ 
ple  of  that  defect  was  also  included  in  close  proximity  to  the  fault  to  provide  a  posi¬ 
tive  form  of  test  control.  Any  failure,  or  evidence  of  fatigue  of  the  correctly  manu¬ 
factured  examples,  would  indicate  overly  severe  environmental  exposure  and  dictate 
that  a  less  stringent  test  be  developed.  No  failures  occurred,  nor  was  there  any 
evidence  of  fatigue  damage  of  any  of  the  good  examples  inserted. 

•  LRU  Level  -  during  the  rapid  thermal  cycling  tests  conducted,  a  total  of  10 
defects  were  detected.  The  application  of  random  vibration  detected  an 
additional  five  faults.  It  was  interesting  to  see  that  the  fault  detect  effi¬ 
ciency  trends  shown  in  Fig.  27  follows  exactly  the  same  behavior  patterns 
measured  during  the  previous  independent  rapid- thermal  cycling  and  random 
vibration  investigations.  That  is,  after  approximately  40  to  50  hours  of 
rapid  thermal  cycling  and  after  four  to  10  minutes  of  random  vibration,  any 
additional  testing  was  not  cost  effective.  It  should  be  noted  at  this  time 
that  the  fault  types  for  this  investigation  were  selected  predominantly  for 
their  thermal  sensitivity.  However,  for  certain  types  of  defects,  tempera¬ 
ture  and  vibration  inputs  each  provide  a  certain  degree  of  stimulation.  The 
increase  in  defects  due  to  vibration  becomes  significant  when  viewed  in  the 
the  above  comments,  and  a  test  conducted  utilizing  both  random  vibration 
and  rapid  thermal  cycling  could  conceivably  be  run  in  a  shorter  time  period 
with  the  same  effectiveness  achieved.  Since  thermal  cycling  costs  "drive" 
total  acceptance  test  costs,  a  substantial  savings  would  be  realized 

•  PCB  Level  -  testing  was  performed  at  the  PCB  level  to  determine  if  thermal 
screening  efficiency  would  be  better  than  at  the  LRU  level.  That  testing 

was  limited  to  thermal  cycling  vrly.  The  results  obtained  during  this  testing 
phase  did  not  follow  any  of  the  previously  established  patterns.  Only  seven 
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Figure  27.  BI»ck-box  level  test  rwultt. 


defects  were  disclosed,  six  within  the  first  10  hours.  However,  all  defects 
were  detected  on  the  two  adjacent  boards  at  the  front  of  the  test  "housing." 
The  boards  were  rotated  to  investigate  the  uneven  failure  distribution  and 
an  additional  seven  hours  of  rapid  thermal  cycling  was  applied,  but  no 
additional  faults  were  detected.  Although  the  uneven  failure  distribution 
could  not  be  explained,  the  fact  that  the  total  number  of  failures  detected 
was  significantly  less  than  those  disclosed  during  LRU  tests  indicates  that 
rapid  thermal  cycling  testing  at  this  level  should  not  be  considered  in  lieu  of 
LRU  level  tests.  Apparently  the  thermal  delta  between  faults  and  surround¬ 
ing  air  is  not  as  great  when  boards  are  directly  exposed  to  temperature 
cycling  as  that  when  the  boards  are  contained  within  a  black-box  enclosure. 
As  a  result,  thermal  stresses  which  produce  physical  stresses  are  not 
adequately  developed.  As  the  years  progressed,  further  investigations  were 
conducted  to  continue  to  measure  the  efficiency  of  ESS  testing  as  it  related 
to  the  sequence  of  environmental  application.  The  results  of  these 
investigations  are  summarized  in  Table  11. 
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TABLE  11.  Environmental  expowr*  optimization  effort*. 


YEAR 

ENVIRONMENTAL  EXPOSURE 

%  EFFECTIVE 

1976 

STANDARD  THERMAL  CYCLING  (168  HOURS) 
RAPID  THERMAL  CYCLING  (168  HOURS) 

10 

30 

1977 

RAPID  THERMAL  CYCLING  (50  HOURS)  1 

THEN  1 

RANDOM  VIBRATION  (10  MINUTES)  J 

42 

1978 

RANDOM  VIBRATION  (5  MINUTES)  ] 

THEN 

RAPIO  THERMAL  CYCLING  (50  HOURS) 

THEN 

RANDOM  VIBRATION  (5  MINUTES 

► 

60 

1979 

RANDOM  VIBRATION  (10  MINUTES)  ] 

THEN  ! 

RAPID  THERMAL  CYCLING  (50  HOURS)  i 

47 

R87-3772-027fT) 

•  Military  Standard  Development  -  the  culmination  of  all  the  latter  Grammar? 
investigations  was  realized  when  MIL-STD-2164(EC) ,  Military  Standard,  En¬ 
vironmental  Stress  Screening  Process  for  Electronic  Equipment,  was  devel¬ 
oped  in  April  1985  for  the  Navy  Electronics  Command.  It  should  be  noted 
that  the  random  vibration  and  rapid  thermal  cycling  requirements  had  been 
successfully  applied  at  various  stages  on  Navy  avionic  procurements  since 
1972. 

The  significance  of  these  new  ESS  requirements  is  summarized  in  the 
Standard's  Forward:  "The  current  emphasis  on  reliability  and  hardware  de¬ 
sign  'jitegrity  has  resulted  In  an  increased  potential  for  providing  a  basically 
sound  and  inherently  reliab'^  design.  As  this  potential  has  increased,  so 
has  the  complexity  and  density  of  packaging  of  contemporary  electronic 
equipment.  This  complexity  and  density  amplifies  the  ever  present  prob¬ 
lems  of  detecting  and  correcting  latent  manufacturing  defects.  The  occur¬ 
rence  of  a  malfunction  due  to  poor  workmanship  incurs  extremely  high  main¬ 
tenance  costs  after  the  equipment  has  been  deployed.  The  fact  that  the  unit 
had  been  fully  qualified  and  demonstrated  a  contractual  mean  time  between 
failures  in  the  laboratory  becomes  meaningless  when  such  a  failure  results  in 
loss  of  life  or  mission. 

Specifications,  standards  and  guidelines  currently  exist  for  development, 
and  qualification  testing.  No  similar  documentation  exists  for  the  Environ¬ 
mental  Stress  Screening  (ESS)  Process;  consequently,  gross  inconsistencies 
in  approach,  coupled  with  test  ineffectiveness,  result  in  latent  defects  caus¬ 
ing  failures  in  delivered  equipment.  This  standard  defines  the  approach  and 
method  to  be  used  for  Environmental  Stress  Screening  of  electronic  equip- 
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ment  so  that  latent  defects  may  be  located  and  eliminated  before  the  equip¬ 
ment  is  accepted.” 

The  intended  application  of  MIL-STD-2164(EC)  is  for  use  on  newly  developed 
equipment.  The  generic  test  levels  and  durations  are  included  in  the  design  re¬ 
quirements  analyzed  by  the  equipment  manufacturer.  The  designer  of  an  electronic 
package  has  to  take  into  consideration  all  of  the  static  and  dynamics  loads  associated 
with  operation,  accelerated  environmental  testing,  reliability,  storage,  shipping,  and 
ESS  acceptance  testing.  Given  due  consideration  to  the  above  requirements,  an 
equipment  designer  can  produce  an  LRU  design  that  will  fulfill  the  contractual 
obligations  of  the  procuring  activity,  without  the  need  to  tailor  environmental 
requirements. 

Therefore,  utilizing  the  above  phi!  sophy,  we  have  demonstrated  on  production 
equipment  that  it  can  be  subjected  to  the  MIL-STD-2164(dC)  requirements  without 
experiencing  any  structural  or  major  operational  performance  problems. 

5.5  STUDY  RESULTS 

As  a  result  of  the  ESS  benefits  realized  on  avionic  equipment,  Grumman  was 
able  to  introduce  the  new  random  vibration  and  rapid  thermal  cycling  screening  en¬ 
vironments  to  several  in-house  aircraft  contracts.  Because  of  the  contractual  prob¬ 
lems  of  the  five  LRUs  evaluated,  only  two  were  exposed  to  random  vibration,  but  all 
five  were  subjected  to  the  rapid  thermal  cycling.  In  any  event,  it  was  possible  to 
measure  the  affects  of  the  new  ESS  requirements  on  equipment  already  in  field  en¬ 
vironment  which  had  been  subjected  to  non-ESS,  i.e.,  sinusoidal  vibration  and  slow 
thermal  cycling,  acceptance  tests.  Then  this  same  type  of  equipment  in  a  subse¬ 
quent  contract  was  exposed  to  the  new  ESS  levels.  A  comparison  of  these  ESS  test 
characteristics  is  described  in  Table  2  of  Subsection  3.1. 

As  a  part  of  this  investigation  a  complete  analysis  of  the  Navy  failure  rate  and 
maintenance  actions  reports  for  these  equipment  was  made.  As  described  in  Section 
3,  the  data  included  over  10  years  of  activity  of  the  non-ESS  and  five  years  of  the 
ESS  on  the  identical  type  equipment.  For  comparison  purposes,  an  analysis  was  per¬ 
formed  on  all  five  LRUs  showing  the  number  of  total  actions  for  the  last  two  years  of 
assessment  for  both  the  non-ESS  and  ESS  equipment.  The  results  (Table  12)  clearly 

show  a  positive  reduction  in  removals  when  ESS  was  applied  for  every  LRU  examined 
in  this  study  by  a  substantial  margin. 
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TABLE  12.  ESS  rff«ctivtn*u. 


EQUIPMENT 

MON-ESS 
MFHBR  | 

ESS 

MFHBR 

% 

IMPROVEMENT 

A 

32 

70 

119 

B 

66 

127 

92 

C 

83 

1*9 

80 

D 

92 

278 

202 

E 

670 

1110 

9S 

«»?-3772-e2fl(T) 


5.6  ANALYSIS  OF  RESULTS 

The  advantages  of  applying  the  generic  environmental  stress  screening  have 
been  well  documented.  Within  Grumman  and  at  our  subcontractor's  facilities,  use  of 
the  generic  levels  and  durations  are  working  very  successfully  on  all  new  design 
electronic  equipment  procurements. 

For  this  study  because  the  potential  ESS  candidate  equipments  in  all  probability 
were  never  designed  for  random  vibration  or  rapid  thermal  cycling,  some  form  of  tai¬ 
loring  must  be  considered.  It  should  be  pointed  out  that  our  experience  has  taught 
us  that  this  does  not  mean  the  equipment  is  incapable  of  withstanding  the  latter  en¬ 
vironment.  In  most  situations  this  same  equipment  probably  has  been  operating  in 
this  type  of  environment,  i.e..  In  a  jet  aircraft  experiencing  rapid  temperature 
changes  and  random  vibration,  without  any  structural  damage. 

In  some  cases,  performance  anomalies  (such  as  out  of  tolerance  conditions)  did 
exist  when  the  generic  levels  were  applied  to  some  of  the  early  vintage  electronics. 
It  became  apparent  after  instrumenting  these  test  units,  that  if  the  response 
acceleration  exceeded  an  amplification  factor  of  10,  a  performance  anomaly  would 
become  evident.  In  these  instances  it  was  necessary  to  tailor  the  random  vibration 
levels  by  notching  at  certain  frequencies  to  minimize  the  operational  problems 
associated  with  marginal  component  or  a  subassembly  installation.  An  actual  example 
of  this  technique  for  a  typical  LRU  with  instrumentation  installed  at  various  locations 
is  described  in  detail  as  follows: 

(1)  Conduct  a  ±3g  sine  sweep  and  measure  the  ratio  (response/control)  at  the 
desired  locations  (see  Fig.  28  for  typical  response) 


VIBRATION  SURVEY  RUN  82-30  SINE  SWEEP  - 
TRANSMISSIBIUTY  PLOT  OP  ACC  9  -  A3000  TRAY 
IN  2  AXIS 
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Ftgurt  28.  Vibration  tunrty  ±.3g  tin*  iw— p  tranimlithrity  plot  in  2  axil. 


(2)  Record  measured  vibration  amplification  factors  exceeding  10,  e.g.. 


Accelerometer  No. 

1 

3 

9 

10 

13 

7 


Location 

Bottom  of  A1000  Assy 

A4000  Tray 

A3000  Tray 

A8000  Tray 

Top  of  A6000  Assy 

ASOOO  Assy 


Amplitude-3G  Input 
11  @  104  Hz 
16  @  104  Hz 

15  @  80  Hz 
20  @  100  Hz 

16  @  103  Hz 
10  @  196  Hz 


(3)  Calculate  the  required  random  vibration  notching  to  reduce  amplifical 
to  10,  e.g. , 

79  to  95  Hz  -  Max  Ampl  =  15  PSD  =  (10/15)  x  .04  =  .0267  GSQD/E 

95  to  104  Hz  -  Max  Ampl  =  20  PSD  =  (10/20)  x  .04  =  .0200  GSQD/P 

104  to  113  Hz  -  Max  Ampl  =  14  PSD  =  (10/14)  x  .04  =  .0286  GSQD/E 

Above  113  Hz  -  Ho  notching  done  in  this  freq  range. 

(4)  Incorporate  notching  of  the  generic  6.0G  RMS  random  vibration  test  s 
trum  as  in  Fig.  29: 
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Figure  23.  Notching  of  random  vibration  test  spectrum. 


With  respect  to  the  rapid  thermal  cycling  requirements,  the  only  tailoring 
necessary  is  to  set  the  temperature  limits  of  the  chamber  so  as  not  to  exceed  the 
operating  temperature  at  which  the  test  article  was  environmentally  qualified. 


Due  to  the  numerous  analytical  ESS  studies  and  tests  conducted,  we  have  de¬ 
veloped  Table  13,  which  summarizes  the  recommended  ESS  regimen  to  be  used  for 
field  equipment.  These  guidelines  are  based  on  tailoring  the  levels  and  procedures 
described  in  MIL-STD-21S4(EC)  and  should  be  directly  applied  to  either  the  LRU  or 
drawer  level  of  assembly.  Our  field  equipment  ESS  investigations  indicated  that  there 
was  little  advantage  to  perform  both  a  40  hour  pre- defect  free  and  a  40  hour  defect 
free  thermal  cycling  test.  The  test  data  indicated  that  the  previous  use  environment 
stimulated  the  major  workmanship  and  man'.ifacturing  problems,  substituting  the  need 
to  do  a  pre-defect  free  period.  To  satisfy  the  defect-free  requirement,  it  was  thus 
decided  that  if  the  candidate  equipment  experienced  an  ESS  failure  during  thermal 
cycling  and  it  was  replaced  immediately  after  that  cycle,  there  should  be  an  adequate 
number  of  defect  free  cycles  accrued  within  the  fixed  40  hour  period  to  satisfy  the 
defect-free  requirement.  This  considerably  reduces  the  test  time  and  its  associated 
costs. 


TABLE  13.  Recommendad  ESS  guidelines. 


RANDOM  VIBRATION 

•  POWER  SPECTRAL  DENSITY 

•  AXES  STIMULATED 

•  OURATION  OF  VIBRATION 

•  POWER  ON  (EQUIPMENT  OPERATION) 

•  EQUIPMENT  MONITORING 

THERMAL  CYCLING 

•  TEMPERATURE  RANGE 

•  TEMPERATURE  RATE  OF  CHANGE 

•  TEMPERATURE  DWELL  DURATION 

•  THERMAL  CYCLING  DURATION 

•  POWER  ON  (EQUIPMENT  OPERATING) 

•  EQUIPMENT  MONITORING 


20-80  HZ  9  +  3dB/0CTAVE 
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6  -  ECONOMIC  ANALYSIS 


6.1  INTRODUCTION 

The  application  of  ESS  is  universally  recognized  as  a  major  technical  expedient 
to  remove  workmanship  and  manufacturing  anomalies  m  electronic  equipment.  The 
associated  costs  to  perform  ESS  testing  on  new  equipment  procurements  is  generally 
easy  to  justify, primarily  because  the  testing  at  the  manufacturer's  facilities.  At  that 
point  in  the  development  and  eventually  in  production,  all  of  the  assets  required, 
i.e.,  technical  expertise,  support  equipment,  manufacturing  facilities  and  environ¬ 
mental  test  equipment,  are  available  on  site.  However,  once  the  electronics  are  sold 
off  to  the  customer,  the  only  resource  he  has  is  to  send  it  back  to  the  manufacturer 
while  still  under  warranty  or  to  repair  it  himself.  Therefore  the  emphasis  is  to  do 
as  much  screening  upfront  as  possible  to  insure  the  reliability  in  that  equipment  is 
what  it  should  be. 

In  the  case  of  non -screened  equipment,  the  problem  the  customer  faces  is  how 
he  economically  justifies  performing  ESS  on  the  thousands  of  electronic  equipments  in 
inventory.  To  this  end,  the  rationale  discussed  in  the  preceding  chapters  of  this 
study  recommends  that  the  solution  is  not  to  conduct  ESS  on  all  the  electronics 
inventory.  The  solution  is  to  select  only  those  equipments  which  can  be  shown  *u 
have  the  highest  potential  for  containing  workmanship  type  defects.  This  selection 
criteria  utilizes  the  maintenance  Information  reported  in  AFM-66-1,  and  permits  the 
user  to  determine  where  he  should  direct  his  engineering  assets. 

The  next  step  in  the  selection  process  is  to  determine  the  ESS  facility,  environ¬ 
mental  equipment,  monitoring,  and  associated  manhours  to  be  able  to  see  the  whole 
picture  before  the  final  decision  can  be  made.  Thus  the  objective  is  to  develop  a 
procedure  to  determine  the  ROI  for  those  electronic  equipments  selected  as  ESS  test 
candidates.  The  cost  methodology  developed  is  described  in  Fig.  30  and  shows  the 
major  ingredients  required  to  determine  if  there  is  truly  a  cost  benefit,  i.e.,  ROI  is 
adequate. 
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Figure  30.  Methodology  for  economic  analysis  of  ESS. 


The  financial  return  on  ESS  is  the  logistics  support  cost  savings,  which  is  th 
difference  between  non-ESS  maintenance  costs  and  those  after  ESS  during  the 
remainder  of  the  life  of  the  LRUs/SRUs  to  be  tested.  Logistic  support  costs  are  th 
result  of  a  complex  maintenance  process  the  majority  of  which  consist  of  the  cost 
associated  with  the  following: 

(a)  Spares  required  to  replace  failed  units 

(b)  Depot  level  repairs 

(e)  Maintenance  labor  both  on  and  off  the  primary  system. 


Figure  31  provides  the  basic  formula  for  determining  the  ROI  and  is  the  resul 
of  conducting  numerous  cost  benefit  trade-off  studies  which  will  be  discussed  in 
detail  in  the  subsequent  paragraphs. 


6  2  LOGISTIC  SUPPORT  COST  SAVINGS  (LSCS 


The  application  of  ESS  results  in  the  reduction  of  downstream  Logistic  Suppor 
Costs  (LSCs)  on  LRUs  and  SRUs  durir-  the  remainder  of  their  life.  This  is  due  t< 
the  expected  decreased  failure  rate  an~  improved  reliability.  ESS's  primary  functio 
is  to  expose  workmanship  and  related  defects  thus  reducing  the  removal  rate  (or 
MTBM)  of  the  units  which  have  masked  the  inherent  design  failure  rate.  The 
amount  of  logistic  support  cost  is  determined  by  the  type  of  maintenance  scenario 
(e.g.,  fighter  aircraft,  transport,  stationary  ground  equipment,  mobile  ground 


WHERE:  ROI 


RS7-3  772-032  (T) 


LSCS  -  (AFC  +  TC) 

I  AFC  +  TC)  (SL-EA! 


X  100  >  33.3% 


ANNUALIZED  RETURN  ON  INVESTMENT 


LSCS  -  LOGISTIC  SUPPORT  COST  SAVINGS 

AFC  -  AMORTIZED  FACILITY  (ENVIRONMENTAL 

&  FUNCTIONAL  TEST)  COST 

TC  -  TEST  COST  (LABOR  TO  PERFORM 

ENVIRONMENTAL  &  FUNCTIONAL  TESTS) 

SL  -  SERVICE  LIFE  OF  LRU/SRU 

EA  -  EQUIPMENT  AGE  AT  ESS 

33.3%  -  MINIMUM  ROI  TO  OBTAIN  3-YEAR  RECOVERY 
OF  ESS  INVESTMENT 

Figure  31.  Return  On  Investment  (ROI)  definition. 


equipment,  etc) .  Field  operational  conditions  and  the  maintenance  scenario  change 
for  different  types  of  avionic  and  electronic  equipment.  The  LRU/SRU  logistic  sup¬ 
port  costs  are  a  function  of  the  maintenance  concept,  i.e.,  whether  repairs  are  be¬ 
ing  done  at  organizational,  intermediate,  depot,  or  factory  levels,  the  discard  p.ulo.> 
ophy,  available  personnel  skill  levels,  etc. 


6.2.1  Logistic  Support  Cost  (LSC)  Development 


In  order  to  develop  the  projected  LSC  before  and  after  ESS  as  well  as  the  ex¬ 
pected  savings  due  to  ESS,  it  is  necessary  ta  utilize  a  methodology  or  model  which 
realistically  simulates  the  specific  logistic  support  scenario  under  which  the  LRUs 
and  SRUs  will  be  maintained.  Furthermore,  the  model  must  be  sensitive  to  the 
changes  induced  by  ESS,  i.e.,  changes  in  MTBM/removal  rate  so  that  realistic  com¬ 
parative  costs  can  be  obtained. 

A  typical  model  methodology  for  determining  projected  logistic  support  cost  is 
the  USAF  Logistic  Support  Cost  (LSC)  Model,  Version  1.1,  dated  January  1979. 
This  widely  used  model  was  developed  for  avionic  systems  and  could  be  readily 
adapted.  It  uses  approximately  50  input  variables  describing  the  maintenance  system 
scenario  and  approximately  25  input  variables  describing  each  LRU.  The  typical 
data  scenario  for  these  inputs  are  provided  in  Appendix  A.  The  input  data  for  the 
model  is  obtained  from  sources  such  as  Air  Force  AFM  66-1  maintenance  data.  Air 
Force  AFLCP  173-10  "AFLC  Cost  and  Planning  Factors  and  AF  Regulation  173-13"  and 
"USAF  Cost  and  Planning  Factors."  The  key  parameters  driving  LSC  are  Mean 
Flight  Hours  Between  Maintenance  (MFHBM)  and  the  unit  ccst  of  the  LRUs. 


Table  14  illustrates  the  application  of  this  model  as  applied  to  the  five  case 
history  LRUs  used  to  demonstrate  the  effectiveness  of  ESS  in  Sections  3,  4,  and  5. 
The  improvement  due  to  ESS  in  the  removal  rates  expressed  as  MFHBR  and  the  re¬ 
sulting  LSC  savings  are  shown.  Since  these  units  were  tested  on  an  as  new  basis, 
the  projections  were  evaluated  over  a  15  year  life  cycle  period.  The  savings  are 
particularly  significant  for  those  units  having  the  lower  MFHBR  rates,  while  those 
with  higher  rates  and  lower  unit  costs  provide  marginal  cost  improvement. 

TABLE  14.  Gross  direct  logistic  support  cost  savings. 
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NOTES: 

•  AF  LSC  MODEL,  VERSION  1.1 .  JANUARY  1979;  OPERATIONAL  DATA  IN  APPENDIX  A 

•  1985  DOLLARS 

•  SERVICE  LIFE:  15  YEARS 

•  BASED  ON  15  YEAR  LIFE  CYCLE 
__________ 

By  parametrically  varying  the  Mean  Time  Between  Maintenance  (MTBM)  parame- 
U  .  ratios  from  1:1  to  2:1,  and  exercising  the  LSC  model,  logistics  support  cost 
savings  as  a  function  of  MTBM  (converted  to  removals  per  month)  were  developed 
and  normalized  as  a  function  of  the  unit  cost  to  provide  a  logistic  support  cost 
savings  factor.  These  relationships  are  provided  in  Fig.  32.  A  service  life  basis  of 
25  years  was  used  since  this  is  the  nominal  average  life  of  a  typical  weapon  system. 


The  LSC  savings  factor,  a  function  of  the  materials  and  labor  cost  savings,  can 
be  expressed  as  the  number  of  equivalent  spares  that  would  bo  saved  per  unit  LRU 
over  the  life  cycle.  Multiplying  the  factor  by  the  unit  cost  would  provide  the  dollar 
cost  savings.  For  example,  if  an  LRU  had  a  removal  rate  of  one  per  month  and  an 
improvement  of  70«  (MTBM  improvement  factor  of  1.7)  was  estimated,  then  the  cor¬ 
responding  LSC  savings  factor  from  Fig.  32  would  be  approximately  three  or  three 
equivalent  spares  saved.  The  corresponding  cost  savings  in  dollars  would  be  three 
times  the  unit  cost  per  LRU,  and  the  total  cost  savings  for  (N)  LRUs  would  be  three 
times  the  unit  cost  x  (N)  over  the  life  of  the  LRU. 
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R86-220i52°GT)  Figura  32.  Projected  field  ESS  logistic  support  cost  saving?  factor 

<s  MTBM  improvement  factor. 

Although  these  characteristics  miy  vary  as  a  function  of  the  model  scenario, 
they  are  fairly  indicative  of  the  orders  of  magnitude  potential  that  could  be  achieved 
for  planning  and  trade-off  purposes. 

6.2.2  Effect  of  Age  on  LSC  Savings 

Logistic  support  cost  savings  are  a  function  of  the  age  of  the  LRU/SRU  at 
which  ESS  is  performed,  irrespective  of  the  stress  effects.  The  savings  will  be 
greater  for  the  younger  equipment  as  a  result  of  two  factors: 

•  If  ESS  were  to  be  performed  when  the  LRU/SRU  is  new  in  the  inventory  the 
unit  will  have  many  more  years  of  life  left  during  which  a  savings  occurs 

•  The  MTBM/removal  rate  will  normally  improve  with  age  as  operational  hours 
are  accumulated  because  defects  will  be  gradually  uncovered  and  repaired. 
The  projected  characteristic  age  curve  developed  in  Section  4,  Fig.  17,  is 
superimposed  on  Fig.  32  to  reflect  the  potential  impact  to  LSCS  due  to  the 
age  of  the  LRU  at  the  time  ESS  is  applied. 
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6.3  ESS  IMPLEMENTATION  COSTS 

ESS  implementation  costs  consist  of  amortized  facility  costs  and  recurring  test 
costs. 

6.3.1  Amortized  Facility  (Non-Recurring)  Costs 

The  formulation  for  amortized  test  facility  cost  is  described  in  Fig.  33.  Each 
factor  in  the  equation  must  be  determined  by  conditions  existing  at  the  facility  where 
ESS  is  to  be  performed  as  well  as  the  quantity  and  time  schedule  for  LRU/SRU 
testing.  In  establishing  the  facility  requirements  costs,  test  duration  and  system 
loading  requirements  must  be  defined.  Typical  average  values  of  the  cost  of 
environmental  equipment  and  set-up  are  listed  in  Table  15.  Cost  of  such  equipment 
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Figure  33.  Non-recurring  amortized  test  facility  cost. 


TABLE  15.  Typical  costs  ot  en»rcnmental 
equipment. 
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10 
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26 
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3 

SINGLE  INSTALLATION  COST 
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176 

cannot  be  standard*  zed  due  to  the  diverse  competitive  nature  that  has  developed  in 
recent  years,  both  economically  as  well  as  technically.  The  costs  are  representative 
of  the  latest  state-of-the-a~t  test  equipment  supportive  of  conducting  typical  ESS 
testing  per  MIL-STD-2164(EC)  and  R&M  2000. 

To  illustrate  the  application,  consider  the  following  criteria: 

•  Number  of  test  hours  available  per  month  (T) :  480 
(5  Days/wk  x  24  hrs/day  x  4  wks/mo) 


•  Test  hours  per  unit  (t)  :  120 

•  Total  Units  to  be  tested  (N)  :  216 

•  Test  equipment  cost/set-up  (C)  :  176,000 

•  Number  of  test  programs  (K)  :  1 

•  Units  flow  through  per  month  (n)  :  24 


The  amortized  test  facility  cost  per  unit  would  then  be: 

AFC  =  ill  (n)  (C)  =  120  (24)  (176,000) 

(T)  (N)  (K)  (480)  (216)  (1) 

AFC  =  $4890/unit 

Therefore  testing  conducted  under  the  above  conditions  is  amortized  at  $4890  per 
unit.  Factors  dropping  the  cost  would  include: 

•  Reducing  the  test  duration;  if  testing  were  to  be  reduced  to  40  hrs/unit, 
the  cost  would  be  reduced  to  $1630  per  unit.  Figure  34  provides  graphic 
illustration  of  test  duration  versus  the  AFC  effects  described  in  the 
illustration 

•  Reducing  the  number  of  test  set-ups  to  handle  the  flows  would  also  be  a 
factor  in  reducing  the  cost  impact.  However,  planning  plays  a  major  roll 
when  you  consider  that  vibration  equipment  would  be  required  only  approxi¬ 
mately  5  minutes  out  of  every  40  hours  of  testing  (8%) ;  this  results  in 
requiring  one  vibration  system  for  every  five  to  10  temperature  chambers. 
Since  vibration  equipment  cost  represents  65%  of  the  total  test  equipment 
cost,  the  amortized  cost  per  facility  set-up  could  be  significantly  reduced  for 
high  density  flowthroughs. 
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Figure  34.  Amortized  facility  cost  per  unit  as  a  function  of  ESS 
test  duration. 


6.3.2  Cost  of  Testing 

Recurring  costs  or  costs  to  physically  perform  the  testing  include: 

•  Cost  of  labor  to  perform  tests 

•  Cost  of  labor  for  electrical  functional  testing  of  LRUs  and  SRUs 

•  Cost  of  spares  to  support  repair 

•  Cost  of  repairs  for  failures  encountered  during  test 

•  Cos*  of  shipping  LRUs  and  SRUs  to  and  from  the  depot. 

Most  of  these  costs  can  be  developed  as  a  function  of  the  logistic  support  cost 
for  the  unit  under  test.  The  testing  will  make  in  effect,  unexpected  logistic  and 
repair  demands,  not  necessarily  planned  for  as  part  of  the  normal  maintenance  rate 
of  the  equipment,  as  well  as  present  demands  for  additional  special  support 
equipment,  as  may  be  needed  to  perform  functional  verification  of  the  equipment 
under  test.  The  cost  of  labor  to  perform  ESS  is  a  function  of  the  test  time  and 
labor  rate  of  personnel  required. 

Figure  35  provides  a  cost  algorithm  for  defining  the  Direct  Test  Cost  (DTC). 
The  cost  is  simply  the  manpower  and  supply  necessary  to  detect,  fault  isolate,  and 
repair  defects  or  failures  encountered  during  the  testing. 
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Figure  35.  Direct  test  cost  algorithim  tor  ESS. 

The  cost  of  labor  and  materials  for  repairs  during  ESS  can  be  calculated  using 
a  variety  of  methods.  One  method  would  be  to  find  an  average  cost  per  repair 
based  upon  historical  experience  with  the  LRU/SRU.  Another  method  (Fig.  35)  uses 
an  LSC  model  to  determine  the  total  logistic  support  cost  over  the  life  of  the 
LRU/SRU  and  the  total  number  of  expected  repairs,  and  dividing  these  to  arrive  at 
a  cost  per  repair.  The  ccst  per  repair  averages  approximately  2.3o  based  on  histor¬ 
ical  experience  (Ref  8)  of  the  spares  unit  acquisition  cost  of  the  LRU/SRU. 

The  expected  number  of  defects  or  repairs  to  be  encountered  during  the  test 
are  a  function  of  the  number  of  removals  per  unit  (p)  or  process  average,  as  de¬ 
fined  in  Section  4.  These  would  be  the  normal  number  of  defects  encountered, 
given  that  ESS  testing  is  stress  dependent  and  accelerates  the  defects  us  they  exist, 
not  as  they  occur  in  time  dependent  scenarios. 

Using  the  average  cost  per  repair  of  2.3°o  of  the  spares  unit  acquisition  cost. 
Fig.  36  was  developed  which  shows  the  repair  cost  during  ESS  versus  the  sparer 
unit  acquisition  cost  for  various  process  averages  (v)  values  of  expected  defect 
repairs . 

It  should  be  noted  that  spares  unit  acquisition  cost  is  the  basis  of  this  calcu¬ 
lation  because  it  is  the  cost  at  which  spare  LRUs/SRUs  are  purchased  and  it  is 
generally  proportional  to  the  complexity  of  the  units,  its  MTBM,  and  to  the  cost  of 
maintenance  of  the  unit. 
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Figure  36.  Direct  test  cost  as  a  function  of  unit  acquisition  cost  & 
process  average  tu) 

6.4  COST  BENEFIT  ANALYSIS 

To  assess  the  cost  benefit  effects  of  applying  ESS  in  a  field  environment,  the 
procedure  defined  in  the  flow  diagram  of  Fig.  37  was  implemented.  Field  data  de¬ 
velopment  was  based  on  an  EF-111A  aircraft  scenario,  and  a  select  number  of  LEUs 
were  used  to  demonstrate  the  process  and  trade-off  techniques.  The  selected  LRUs 
are  as  listed  in  Table  16  which  includes  pertinent  statistical  data  as  applied  in  the 
analyses.  Weapon  system  deployment  and  operational  statistics  are  provided,  with 
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Figure  37.  Procedure  for  economic  »n»lyii», 

appropriate  assumptions,  in  Appendix  A  as  well  as  relevant  LRU  data  as  developed 
from  AF/D056  Data  System  for  the  reporting  period  from  July  1984  through  June 
1986. 

6.4.1  Logistic  Support  Cost  Savings  (LSCS)  Effects 

Field  data  assessment  was  used  to  determine  potential  candidate  LRUs  for  ESS 
using  the  methods  described  in  Sections  3  through  5.  The  LRU  unit  rank  by 
removal  is  provided  in  Appendix  B.  Of  the  six  equipments  selected,  five  were 
pulled  from  the  top  25  ranked  for  high  removals  for  the  reporting  period  (including 
number  1  ranked  MBE  unit),  and  a  sixth  just  below  the  top  25  (the  DDD  ranged 
number  31).  The  parameters  of  the  LRUs  were  inserted  into  the  LSC  model  for 
obtaining  downstream  projected  logistic  support  costs.  These  are  provided  in  Table 


1M6& 


TABLE  16.  LRU  Mltction  prcfita. 


LRU 

SPECIFIED 

MTBF 

OPERATIONAL 

MFH8R 

NUMBER  OF 
UNITS 
REPORTING 

(N) 

UNIT 

COST 

SK 

PROCESS 

AVO 

PER 

UNIT 

<M> 

SERVICE 

LIFE 

YEARS 

HIGH  RMVL 
RATE  RAN 

EXCITER  (VBE) 

(WUC  76ZMO) 

500 

31 

144 

*87 

30 

25 

i 

RF  CALIBRATOR 
(RFC) 

'WUC  76ZPJ) 

1500 

67 

36 

174 

4.2 

25 

7 

RAOAR  IR 
INDICATOR  tRIRI) 
(WUC  73BRO) 

1165 

85 

36 

302 

3.2 

25 

11 

COMPUTER  SYNC 
UNIT  ICSU) 

(WUC  76Y20I 

502 

83 

36 

444 

3.3 

25 

12 

SIGNAL  DATA 
CONVERTER  !SCC! 
(WUC  76Y20I 

848 

133 

36 

401 

1.5 

25 

17 

DIGITAL  DATA 
DISPLAY  (ODD) 

(WUC  76Y10! 

853 

257 

36 

116 

1.1 

25 

31 
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A-4  of  Appendix  A.  The  MFHBR  removal  rates  before  and  after  ESS  were  used  i 
the  LSC  model  to  generate  the  LSC  values  before  ESS,  Lad  then  after  as  a  functio 
of  the  improvement,  based  on  an  initial  maximum  improvement  assumed  at  2:1.  T1 
rates  were  then  parametrically  reduced  as  a  function  of  equipment  age  at  which  it 
assumed  the  LRU  is  being  tested.  This  was  done  using  the  LSCS  generic  curve 
•’  ed  in  Fig.  32.  The  LSC  savings  effects  developed  are  as  shown  in  Table  1 
1  he  assumed  ages  of  the  LRUs  at  the  time  of  ESS  were  varied  to  assess  the  potenl 
parametric  relationship  that  eould  develop.  Overlaying  the  removal  rates  (in  terr 
of  removals  per  month)  onto  the  generic  LSCS  curves  in  Fig.  32  provides  the 
improvement  factor  effects  that  can  be  derived.  The  results  are  graphically 
provided  in  Fig.  38.  It  can  be  seen  irrespective  of  any  other  cost  effect  that  ma 
further  reduce  the  LSCS  due  to  the  test  implementation  costs,  that  the  smaller  tl 
change  in  ;he  removal  mite  the  smaller  the  logistic  support  cost  savings.  The  MBI 
the  highest  ranked  item  and  the  DDD,  the  lowest  ranked  item,  set  the  boundarif 
with  all  other  units  falling  in  the  midrange. 

6.4.2  Implementation  Cost  Effects 

ESS  Implementation  Costs  were  developed  as  a  function  of  the  ESS  test  durati 
extremes: 

•  Assuming  implementation  of  MIL-STD-2164(EC) ,  which  is  40  hours  of  burn-in 


TABLE  17.  Logistic  support  cost  savings  effects  as  a  function  of 


changes  in  removal  rates  (WFHBRL 


LRU 

UNIT 

COST 

s 

AGE 

Ar  ESS 
(YR3) 

MFHBR 
W/0  ESS 

MFHBR 

WESS 

LSC* 

SAVINGS 

FACTOR 

LSC 

SAVINGS 

PER  UNIT 

$ 

MBE 

467000 

0 

30 

60 

4 

1948000 

1 

30 

57 

36 

1753200 

3 

30 

52 

29 

1412300 

5 

JO 

46 

22 

1071400 

10 

30 

40 

1 

487000 

15 

30 

i 

37 

03 

146100 

RFC 

174000 

0 

67 

134 

2 

348000 

1 

67 

128 

18 

313200 

3 

67 

116 

1.6 

278400 

5 

67 

104 

1 

174000 

10 

67 

90 

05 

87000 

15 

67 

82 

0.1 

17400 

RIRI 

302000 

0 

85 

170 

1.4 

422800 

( 

t 

65 

162 

13 

392600 

3 

85 

147 

1 

302000 

5 

85 

131 

0.7 

211400 

10 

85 

113 

03 

90600 

15 

85 

104 

0.1 

30200 

CSU 

444000 

0 

88 

176 

1.4 

621600 

1 

88 

168 

13 

577200 

3 

88 

152 

1 

444000 

5 

88 

136 

0.7 

310800 

10 

88 

117 

03 

133200 

15 

88 

108 

0.1 

44400 

EDO 

401000 

0 

129 

258 

1 

401000 

1 

129 

246 

0.9 

360900 

3 

129 

|  223 

0.7 

280700 

5 

129 

99 

0.5 

200500 

10 

129 

•  > 

0.3 

120300 

15 

129 

143 

0 

0 

POD 

116000 

0 

2S7 

514 

0.6 

69600 

1 

257 

491 

0.5 

58000 

3 

257 

445 

0.4 

46400 

5 

257 

306 

0.3 

34800 

« 

257 

342 

0.1 

11600 

! 

ts 

257 

285 

0 

0 
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•OBTAINED  FROM  FIG.  32  (SUBSECTION  6J.1I 
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and  then  40  hours  failure  free  in  an  80  hour  window  for  a  potential  maximum 
duration  of  120  hours  of  testing 

•  Implementation  of  recommended  duration  reduction  for  field  inventoried 
equipment  of  40  hours,  without  failure  free  as  discussed  in  Section  5. 


The  ESS  implementation  costs  were  developed  using  the  algorithms  for  both  non¬ 
recurring  and  recurring  costs  in  Subsection  6.3.  The  cost  developed  for  each  LRU 
is  provided  in  Table  18. 


TABLE  18.  Development  of  ESS  imolenientttion  colt  par  unit  tetted. 


LRU 

TOTAL* 

LSC 

BEFORE  ESS 
SM 

•TOTAL 

REPAIRS 

PER 

LIFECYCLE 

COST 

PER 

REPAIR 

SK 

PROCESS 
AVG  (*i) 

PE  MV  LS/ 
UNIT 

ttt 

REPAIR 

COST 

DURING 

ESSSK 

••TEST 

FACILITY 

LABOR 

SK 

S  AMORT. 
FACILITY 
COST/UNIT 

TEST 

IMPLEMENT 

COST 

SK 

MBE 

143.2 

13500 

10.6 

3.0 

31.8 

6 

3240 

41.0 

RFC 

24  3 

6018 

4.0 

4.2 

16.0 

6 

3240 

25.2 

RIRI 

32.4 

4765 

6.8 

3.2 

20.4 

6 

3210 

29.6 

CSU 

45.7 

4765 

9.6 

3.3 

28.8 

6 

3240 

38.0 

SDC 

28.0 

3140 

8.9 

1.5 

173 

6 

32'*0 

27.0 

ODD 

4.9 

1576 

3.1 

1.1 

3.1 

6 

3240 

123 

•OBTAINED  FROM  LSC  MODEL 

••TEST  LABOR  COST.  ASSUMES  TWO  PEOPLE  TO  MONITOR  TEST  EQUIPMENT  AT  120  HRS  PER  TEST  AT  S2S/HR 
tttOBTAINED  FROM  FIG.  36  (SUBSECTION  6.3.2)  FOR  PROCESS  AVG  („) 

R67-3772-041(T) 


Amortized  facility  costs  were  developed  assuming  the  flow  rate  of  324  units, 

with  testing  of  all  units  to  be  completed  within  a  year  or  27  units  per  month.  The 

available  test  hours  per  month  were  assumed  at  480.  This  results  in  non-recurring 

AFC  of  $3240/unit  for  the  120  hour  test  and  $1080/unit  for  the  40  hour  test.  A 

comparison  of  the  ESS  implementation  cost  per  unit  for  the  120  hour  and  40  hour  test 

durations  indicates  that  the  ESS  implementation  cost  is  averaging  between  9%  and  7% 

of  the  unit  cost  noted  in  Table  19 .  The  only  costs  that  are  actually  affected  by  the 

TABLE  19.  ESS  implementation  cog  as  a  function  of  unit  eott 


EQUIPMENT 

SK 

UNIT 

COST 

ESS  IMPLEMENTATION  COST  SK 

*  UNIT  COST 

120  HR 

40  HR 

120  HR 

40  HR 

MBE 

PT7 

41 

343 

8% 

7% 

RFC 

74 

26.2 

19.1 

14% 

11% 

RIRI 

302 

294 

23.5 

10% 

8% 

CSU 

444 

38 

313 

9% 

7% 

SOC 

401 

27 

203 

7% 

8% 

DDO 

118 

123 

8.2 

11% 

8% 

AVG 

9% 

7% 

R«7-377J-042(T) 


reduced  test  duration  are  the  AFC  and  the  test  facility  labor  cost.  Repair  costs  will 
be  unaffected.  As  noted  for  the  MBE  in  Table  19,  there  is  little  cost  i.t  ect  due  to 
test  duration  (8°6  for  120  hours  compared  to  7%  for  40  hours).  This  i'  hecai  -e  the 
repair  yield  will  be  high  in  either  event.  For  the  DDD  unit,  the  yield  be  low 
and  the  test  duration  will  now  have  an  effect  (more  than  one  half  the  cc^  ,  ' '%  for 
120  hours  compared  to  5%  for  40  hours) . 

The  resulting  test  implementation  costs  developed  (cost  for  120  hour  testing  was 
used)  were  then  subtracted  from  the  LSCS  developed  in  Table  17,  and  the  net  cr,*t 
savings  and  ROI  were  computed  to  develop  Table  20.  The  cost  savings  rank  pro¬ 
vides  an  overview  indication  of  candidate  equipments  to  be  tested  if  you  had  varying 
conditions  of  age  and  improvement  factors  to  contend  with  in  the  selection  process. 
The  MBE,  by  virtue  of  its  frequency,  eclipses  almost  all  other  candidates  for  all  age 
and  rate  improvement  categories. 

6.4.3  Return  On  Investment  (ROI) 

The  annualized  ROI  was  computed  in  accordance  with  the  formula  in  Fig.  31  and 
these  results  are  shown  in  the  final  column  of  Table  20  for  each  LRU  and  ESS  age. 
The  listings  provide  an  indication  of  the  combinations  that  will  /lave  a  33%  ROI  or 
better  and  are  the  most  likely  candidates.  Significant  notes  include: 

•  Equipment  whose  age  exceeds  10  years  at  a  time  of  evaluation  offers  little  in 
the  way  of  economic  benefit 

•  Contributors  beyond  the  top  25  are  not  cost  effective  as  noted  by  the  DDD 
unit,  which  was  ranked  31  and  could  not  break  33%  under  any  condition. 

Figure  39  (1  through  6)  is  developed  to  assess  the  variation  in  the  ROI  as  a 
function  of  the  efficiency  of  the  test  program,  that  is,  the  percentage  of  defects 
that  are  reduced  and  the  test  duration.  The  graphic  comparisons  are  for  defect 
yield  rates  of  100%  (Fig.  39  (1  and  2)),  50%  (Fig.  39  (2  and  3)),  and  25%  (Fig.  39 
(5  and  6))  considering  the  test  durations  of  120  or  40  hours  as  well  as  the  age  at 
the  time  the  LRU  is  tested.  In  both  test  duration  categories,  as  the  potential  gain 
in  removal  rate  is  reduced,  the  number  of  economically  viable  units  significantly 
reduces  (ROI  drops  below  33%).  Reducing  the  number  of  units  to  be  tested  while 
retaining  a  high  potential  for  defect  reduction,  such  as  the  case  with  selecting  bad 
actors,  would  provide  the  best  economical  combination  (together  with  the  reduced 
test  time)  possible,  as  illustrated  by  Fig.  39  (1). 


TABLE  20.  ESS  projected  cot  ta»ing»  &  return  on  iwmtment. 


LRU 

UNIT 

COST 

$ 

i 

AGE 
AT  ESS 
(YRS) 

MFHBR 
W/O  ESS 

MFHBR 
W  ESS 

LSC* 

SAVINGS 

FACTOR 

LSC 

SAVINGS 
PER  UNIT 

S 

ESS” 

IMPLEMENT 

COST 

$ 

NET 

SAVINGS 
PER  UNIT 

8 

COST 

SAVINGS 

RANK 

ANNUAL 

RETURN 

ON 

.NVESTMT 

% 

«BE 

487000 

0 

30 

60 

4 

1943000 

41000 

1907000 

1 

186 

1 

30 

57 

3.6 

1753200 

1712200 

2 

174 

3 

30 

52 

2.9 

1412300 

1371300 

3 

152 

5 

30 

46 

2.2 

1071400 

1030400 

4 

125 

10 

30 

40 

1 

487000 

446000 

7 

73 

IS 

30 

37 

0.3 

146100 

105100 

22 

26 

RFC 

174000 

0 

67 

134 

2 

384000 

25200 

322800 

13 

51 

1 

67 

128 

1.8 

313200 

288000 

mm 

48 

3 

67 

116 

1.6 

278400 

253200 

46 

5 

67 

104 

1 

174000 

148800 

30 

10 

67 

90 

0.5 

87000 

61800 

16 

IS 

67 

62 

0.1 

17400 

-7800 

-3 

RIRI 

302000 

0 

85 

170 

1.4 

422800 

29600 

393200 

'  i 

53 

1 

85 

162 

1.3 

392600 

363000 

ii 

51 

3 

85 

147 

1 

302000 

272400 

16 

42 

5 

85 

131 

0.7 

311400 

181800 

19 

31 

10 

85 

113 

0.3 

90600 

61000 

26 

14 

IS 

85 

104 

01 

30200 

600 

32 

0 

CSU 

444000 

0 

88 

176 

1.4 

621SOO 

38000 

583600 

61 

1 

88 

168 

1.3 

577200 

539200 

59 

3 

88 

152 

1 

444000 

406000 

49 

S 

88 

136 

0.7 

310800 

272800 

IS 

38 

<0 

88 

117 

0.3 

133200 

95200 

23 

17 

IS 

88 

108 

0.1 

44400 

6400 

31 

2 

SDC 

401000 

0 

129 

258 

1 

401000 

27000 

374000 

10 

56 

1 

129 

246 

0.9 

360900 

333900 

12 

52 

3 

129 

223 

0.7 

280700 

253700 

17 

43 

S 

129 

199 

O.S 

200500 

173500 

20 

32 

10 

129 

172 

0.3 

120300 

93300 

24 

23 

IS 

129 

143 

0 

0 

-27000 

36 

-10 

000 

116000 

0 

257 

514 

0.6 

96900 

12300 

57300 

27 

19 

1 

257 

491 

0.5 

58000 

45700 

28 

16 

3 

257 

445 

0.4 

46400 

34100 

29 

13 

s 

257 

396 

OS 

34800 

22500 

30 

9 

10 

257 

342 

0.1 

11680 

-700 

33 

0 

IS 

257 

285 

0 

0 

•12300 

35 
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Figure  39.  ROI  effect]  as  a  function  of  ESS  teat  efficiency  (sheet  2  of  2). 


Similarly,  in  testing  SRUs  (which  represent  only  a  fraction  of  the  potential  de¬ 
fects  rate),  unless  they  have  a  significant  contribution  of  the  LRU’s  defect  popu¬ 
lation,  the  economic  potential  (as  can  be  derived  from  the  removal  rate  improvement) 
is  not  justifiable,  and  the  potential  is  practically  non-existent  as  the  test  duration 
increases  to  120  hours,  as  illustrated  by  Fig.  39(6). 

6.5  COST  BENEFIT  CONCLUSIONS 


Based  on  the  analyses  conducted,  it  has  been  shown  that: 

•  Equipment  age  should  not  be  greater  than  10  years  at  the  time  ESS  is 
implemented,  ROI  falls  well  below  33%  unless  the  LRU  is  a  significant  con¬ 
tributor  to  the  weapon  system  removal  rate 

•  ROI  of  33%  is  a  good  indication  of  potential  cost  benefit  that  can  be  realized 
as  a  result  of  field  ESS 

•  Unless  unit  under  consideration  is  a  high  removal  rate  contributor  (top  25  or 
better)  cost  benefit  is  well  bel.w  expectation 

•  Reduced  test  duration  (eliminc  t  on  of  failure  free  categorization)  offers  op¬ 
portunity  to  test  more  for  less 


•  High  bad  actor  effectiveness  selection  would  be  the  optimal  cost  benefit  ef¬ 
fect  that  could  be  achieved  (fewest  units  tested,  high  potential  defect  yield) 

•  Lower  level  of  assembly  testing  should  be  able  to  demonstrate  significant  im¬ 
provement  in  removal  rate,  nominally  greater  than  25°  to  insure  effective 
pay-off  of  field  ESS 

•  ESS  test  implementation  costs  based  on  repair  costs  as  a  function  of  logistic 
support  costs  per  repair,  could  average  between  7°  and  9°o  of  unit  cost, 
which  includes  any  amortized  facility  costs.  This  cost  covers  additional 
sparing  and  support  equipment  requirements  resulting  from  the  testing  and 
repair  of  units  selected. 
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7  -  CONCLUSIONS  &  RECOMMENDATIONS 


7.1  FIELD  ASSESSMENT 

•  Comparative  analysis  of  established  case  histories  of  electronic  LRUs  has 
demonstrated,  via  field  history  performance,  effective  improvement  in  total 
removal  rates  attributable  to  ESS.  This  is  concluded  by: 

-  Reduction  of  overall  removal  rates,  on  the  order  of  2:1  across  the  board 
for  ESS  populations 

-  Significant  stability  of  removal  rate  frequencies  and  frequency  levels  for 
ESS  populations  as  compared  to  more  widely  dispersed  rates  for  non-ESS 
populations 

-  Improved  removals  per  unit  by  LRU  serial  number. 

7.2  DATA  REDUCTION 

•  The  grouping  of  data  in  terms  of  Type  1  (performance),  Type  2  (mainten¬ 
ance  induced),  and  Type  6  (false  alarm)  maintenance  actions  provides  some 
intuitive  attributes  that  can  identify  potential  latent  (workmanship)  defects, 
as  compared  to  inherent  (design)  defects.  This  grouping  consists  of: 

-  Latent  characteristic  actions: 

•  Type  1  maintenance  actions  resulting  in  repair  without  parts 

•  Type  2  maintenance  reduced  actions 

•  Type  6  false  alarms  (or  cannot  duplicate) 

-  Inherent  characteristic  actions: 

•  Type  1  maintenance  actions  resulting  in  repair  with  parts 

•  NRTS  (Not  Repaired  This  Station)  -  hardware  returned  to  depot  for 
further  disposition 

•  Assessment  of  bad  actor  serialized  LRUs  provided  significant  insight  into  the 
behavior  pattern  of  these  actions.  As  noted  in  Table  21,  the  percent  in 
removal  rate  gain  affected  by  the  bad  actor  selection,  shows  Type  6  actions 
gaining  at  a  rate  of  about  2:1  in  contrast  to  either  of  the  Type  1  conditions 
(with  parts  or  without  parts).  This  is  highly  indicative  of  the  elimination  of 
intermittent  defects  that  tend  to  create  false  alarms.  Type  classification  and 
the  organization  of  maintenance  actions  in  the  data  reduction  process. 


TABLE  21.  Percent  gain  distribution  in  removal  rate  as  function  of  bad  actor  removal., 


INHERENT 

LATENT 

EQUIPMENT 

TYPE  1  WITH 
PARTS  % 

TYPE  1 

W/O  PARTS  % 

TYPE  2  &  TYPE  6 
% 

A 

13% 

35% 

46% 

B 

21% 

10% 

49% 

C 

20% 

7% 

43% 

D 

17% 

26% 

20% 

E 

11% 

20% 

62% 

•WEIGHTED  AVERAGE 

16% 

21% 

44% 

WEIGHTED  AVERAGE  AS  A  FUNCTION  OF  REMOVAL  RATE 


P8  7-3  772-0  74  (T) 


therefore,  do  offer  potential  for  assessing  ESS  effects  from  a  field  reporting 
point  of  view.  But  once  again,  these  can  only  support  circumstantial 
conclusions,  since  inherent  and  latent  characteristics  are  extremely  difficult 
to  quantify  without  supporting  failure  analysis  data. 

7.3  LEVEL  OF  ASSEMBLY  SENSITIVITY 

•  In  the  field  scenario  the  LRU  provides  the  collection  point  for  quality  related 
problems.  It  is  the  focal  point  of  all  field  maintenance  reporting  and 
provides  reasonable  traceability  to  establish  quality  characteristics,  including 
measurable  removal  rates  and  individual  unit  traceability  by  serial  number 

•  The  SRUs  and  other  lower  levels  of  assembly  offer  no  traceability  and  re¬ 
quire  accountability  through  the  cognizant  ALC.  Since  the  ALC  is  not  the 
assembly  point  of  the  LR7',  there  is  no  process  control  that  can  be  affected 
through  the  screening  o.’’  the  lower  levels  of  assembly.  Further,  SRU  ana 
lower  level  component  screens  are  hignly  sensitive  to  the  contribution  the 
level  hardware  is  actual’}  making  to  the  LRU  removal  rate.  Selection  at 
lower  levels  for  field  screening  must  be  carefully  assessed  to  establish  that 
‘he  screen  will  have  an  impact  on  the  LRU  on  a  per  repair  basis.  This  is 
reflected  by  the  fact  that,  for  the  case  histories  studied,  neither  SRUs, 
lower  levels  of  assembly,  nor  piece  parts  were  screened  other  than  as 
required  per  the  component  specifications,  and  the  LRU  ESS  population  at 
their  current  repair  rates  have  not  shown  any  degradation  over  a  period  of 
five  years. 
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7.4  SELECTION  CRITERIA 


•  High  removal  contributor  ranks  provide  the  prioritization  of  equipment  and 
LRUs  to  initiate  the  process  of  picking  candidate  hardware  for  screening. 
This  is  made  possible  by  the  fact  that  the  total  number  of  equipment  remov¬ 
als  from  a  weapon  system  is  nominally  contributed  by  a  relatively  small 
percentage  of  the  total  population  of  electronic  equipment  making  up  the 
system.  Unless  the  candidate  LRU  is  a  high  removal  contributor  initially, 
the  effect  field  ESS  can  have  on  the  aggregate  removal  rate  is  minimal 

•  Comparisons  of  removal  rates  (MTBR)  to  predicted  or  specified  reliabilities 
(MTBF)  provide  some  insight  to  potential  candidates  in  that  the  latent  type 
defects  will  suppress  the  true  failure  rate.  This  is  particularly  significant 
for  small  population  systems,  e.g.,  ground  radars,  ground  test  equipment, 
etc  where  processes  such  as  bad  selection  is  not  feasible.  The  discrimina¬ 
tion  ratio  of  MTBF  >  .  provides  the  indication  of  potential  candidate  selec- 

MTBR 

tion.  The  greater  the  magnitude,  the  greater  the  possibilities. 

7.5  BAD  ACTOR  SELECTION 

•  Bad  actor  selection  provides  a  process  for  selecting  a  small  number  of  LRUs 
from  the  high  contributor  population,  identifiable  by  serial  number,  which 
provide  the  highest  percentage  of  ESS  sensitive  defectives.  It  is  concluded 
that  this  approach  warrants  further  investigation,  since  it  identifies  specific 
units  having  frequencies  higher  than  the  operational  norms.  This  is 
particularly  significant  when  you  consider  that  the  pulling  LRUs  from  the 
field  for  testing  is  highly  undesirable,  from  a  field  readiness  and  equipment 
availability  point  of  view.  It  is  recommended  that  pilot  programs  be  estab¬ 
lished  to  access  the  feasibility  and  effectiveness  of  such  a  program. 

7.6  EQI  IPMENT  AGE 

•  Equipment  age  and  growth  effects  cannot  be  clearly  quantified.  Extrapola¬ 
tion  and  averaging  of  growth  experience  curves  of  the  five  case  histories 
tend  to  indicate  that  LRUs  tested  beyond  10  years  of  age  offer  little 
improvement  Denefit  as  a  result  of  screening.  This  cannot  be  supported  by 
the  study's  case  history  ESS  population  since  all  equipments  tested  were  new 
and  not  field  deployed.  The  nature  of  age  effects  can  only  be  determined  as 
a  function  of  experience  factors  which  up  to  this  time  have  not  been 


available.  This  is  supported  by  existing  army  tailoring  studies  (Ref  9) 
which  indicate  that  a  successful  ESS  program  can  be  implemented  on  over¬ 
hauled  units  of  between  15  and  25  years  in  service. 

7.7  GENERIC  ESS 

•  The  case  study  LRUs  were  all  generic  ESS  tested.  In  all  cases,  as  con¬ 
cluded  from  the  field  performance  assessment,  results  showed  on  the  order 
of  2:1  improvement  over  the  non-ESS  counterparts.  Since  the  field  ESS 
implementation  program  is  planned  for  field  deployed  equipment,  normally 
repaired  at  a  depot,  it  is  recommended  that  a  generic  ESS  profile  as 
typically  defind  by  MIL- STD-2164  (EC)  be  used.  This  will  simplify  ESS  test 
operations,  reduce  ESS  test  equipment  set-up  costs  and  minimize  ESS  train¬ 
ing  requirements.  To  minimize  the  potential  for  stress  over  exposure,  par¬ 
ticularly  to  older  equipment,  it  is  recommended  that  the  failure  free  portion 
of  the  testing  be  reduced,  as  long  as  each  ESS  defect  encountered  is  re¬ 
paired  when  it  occurs.  It  is  expected  that  during  a  full  functional  40  hours 
of  cycling  there  will  be  a  justify  He  amount  of  failure-free  time  to  validate 
the  repair 

•  SRU  screens  should  be  limited  to  spares  and  those  items  which  have  experi¬ 
enced  higher  failure  rates  in  the  field.  The  optimum  environmental  test  in 
these  cases  is  a  non-operating  thermal  shock  defined  in  the  Field  ESS  Imple¬ 
mentation  Guidelines  in  Appendix  C 

•  Tailoring  should  be  considered  in  those  cases  where  the  probability  exists 
that  the  equipment  may  not  have  been  designed  to  function  in  a  generically 
defined  random  vibration  or  rapid  thermal  cycling  environment.  Techniques 
as  defined  in  Subsection  5.6  should  be  considered  to  minimize  the  potential 
problem  for  the  equipment  when  generic  i'SS  levels  are  applied. 

7.8  ESS  ECONOMICS 

•  The  economics  of  ESS  based  on  ROI  (nominally  33-1/3%)  are  effectively 
achieved  most  consistently  by  testing  a  minimum  number  of  units  that  can  be 
selected  to  provide  a  maximum  defective  yield.  Bad  actor  selection  offers 
the  best  opportunity  to  achieve  these  goals.  To  maximize  the  cost  benefit: 

-  Units  selected  should  be  high  removal  rate  contributors.  This  will 
maximize  the  logistics  support  cost  savings 
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-  Reduced  test  durations  (by  eliminating  failure  free)  reduce  test  imple¬ 
mentation  cost  by  minimizing  test  facility  loading 

-  Lower  levels  of  assembly  testing  should  demonstrate  significant  improve¬ 
ment  potential  in  removal  rate,  nominally  greater  than  25°o  to  ensure  ef¬ 
fective  ROI  pay-off 

-  The  combination  of  equipment  service  life  at  time  of  test  and  rate  im¬ 
provement  should  be  such  to  insure  at  least  33°o  ROI. 

7.9  FIELD  ESS  IMPLEMENTATION  GUIDELINE 

•  The  guideline  as  provided  in  Appendix  C  defines,  as  a  minimum,  the  method¬ 
ology  to  develop  an  economically  viable  program  by  implementing  technically 
sound  techniques.  Figure  40  provides  the  recommended  implementation  task 
flow  requirements.  The  ESS  selection  program  recommended  is  divided  into 
four  major  areas: 

(1)  Equipment  Selection  -  based  on  existing  AFD05G  and  MODAS  databases 
and  supporting  ALC  data,  and  establishes  equipment  population  and 
maintenance  rate  histories  sensitive  to  ESS 

(2)  Selection  Criteria  -  establishes  selection  means  to  minimize  the  quantity 
and  quality  of  the  equipment  selected  for  testing 

(3)  Test  Profile  Development  -  obtain  equipment's  environmental  quali¬ 
fication  and  operational  capabilities,  and  apply  generic  temperature  and 
vibration  levels  and  durations,  which  include  test  tailoring  practices 
and  considerations 

(4)  Economic  Selection  -  conduct  cost  studies  to  optimize  facility,  test 
equipment,  manhours,  and  test  requirements  to  accurately  assess  all 
costs  to  determine  if  an  ROI  savings  is  obtainable. 


Figure  40.  Field  ESS  implementation  program  task  flow 
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10  -  DEFW  .ONS  &  ACRONYMS 


10.1  DEFINITIONS  APPLICABLE  TO  THIS  REPORT 

AFD056  -  Output  process  data  tapes  and  reports  compiled  in  accordance  with  D056 
Data  System  directive  for  reporting  AF66-1  Maintenance  Data  Collection. 

Army  UMSDC  -  Unscheduled  Maintenance  Sample  Data  Collection  System,  output  pro¬ 
cess  data  tapes  and  reports  comp'led  in  accordance  with  Army  regulation  TR-216. 

Attributes  -  A  characteristic  or  property  which  is  appraised  in  terms  of  whether  it 
does  or  does  not  exist  (go  or  no  go),  with  rerpect  to  a  given  requirement. 

Bad  Actor  or  Loser  Box  -  Units  of  product  which  have  a  frequency  of  removals 
greater  than  the  norm  or  process  average. 

Consumable  -  An  item  of  material  used  up  beyond  recovery  in  the  use  for  which  it 
was  designed  or  intended. 

Defect  -  Any  non-conformance  of  the  unit  of  product  with  specified  requirements. 
Defective  -  A  unit  of  product  which  contains  one  or  more  defects. 

Discrimination  Ratio  (DR)  -  The  measure  of  steepness  between  specified  MTBF(s)  and 
MTBR  to  discriminate  between  high  and  low  product  defective  potential.  Numerically 

potential 

MTBR 

•  MTBF(s)  >  1;  h.gh  potential 
MTBR 


Environmental  Stress  Screening  (ESS)  -  A  series  of  environmental  tests  conducted  to 
disclose  weak  parts,  workmanship  defects  and  manufacturing  process  anomalies. 


ESS  Test  Profile  -  The  sequence  and  duration  of  stress  environments  under  which 
item  is  to  be  subjected. 

Failure  Free  Time  -  A  contiguous  period  of  time  (in  terms  of  environmental  cycles) 
during  which  an  item  is  to  operate  without  the  occurrence  of  a  failure  while  under 
environmental  stress. 

Failure  Rate  -  Reciprocal  of  MTBF. 

Field  ESS  -  ESS  performed  by  a  cognizant  equipment  ALC,  in  lieu  of  manufacturer  at 
the  manufacturer's  facility. 

Inherent  Defect  -  A  failure  or  defect  that  is  a  function  of  the  intended  design  appli¬ 
cation  of  the  item,  when  operated  in  its  intended  operational  and  logistic  support  en¬ 
vironment. 

Latent  Defect  -  A  process  induced  (manufacturing,  quality,  maintenance)  weakness, 
not  detected  by  ordinary  means,  which  will  either  be  precipitated  by  ESS  screening 
conditions  or  eventually  fail  in  its  intended  use  environment. 

Line  Replaceable  Unit  (LRU)  -  A  unit  normally  removed  and  replaced  as  a  single  item 
which  consists  of  assemblies  (SRUs),  accessories,  and  components  that  collectively 
perform  a  specific  functional  operation. 

Maintenance  Action  -  An  element  of  a  maintenance  event.  One  or  more  tasks  (e.g., 
removal,  fault  detection,  fault  isolation,  repair  and  inspection)  necessary  to  retain 
an  item  in,  or  restore  to,  a  specific  condition. 

Maintenance  Event  -  One  or  more  maintenance  action  required  to  effect  corrective 
maintenance  due  to  any  type  of  failure,  or  malfunction,  false  alarm.  Categorization 
of  maintenance  events  based  on  the  D056  Air  Force  definition  are  as  follows: 

•  Type  1  -  this  code  indicates  that  the  item  can  no  longer  meet  the  minimum 
specified  performance  requirements  due  to  its  own  internal  failure  pattern . 

•  Type  2  -  this  code  indicates  that  the  item  can  no  longer  meet  the  specified 
performance  requirement  due  to  some  induced  condition  and  not  due  to  its 
own  internal  failure  pattern. 


•  Type  6  -  this  code  indicates  that  maintenance  resources  were  expended  due 
to  policy,  modifications,  item's  location,  cannibalization,  or  other  'no  defect' 
conditions  existing  at  the  time  maintenance  was  accomplished. 

Mean  Time  Between  Failure  (MTBF)  -  The  mean  number  of  life  unit  (i.e.,  operating 
hours,  flight  hours,  etc)  during  which  all  parts  of  the  item  perform  within  specified 
limits,  during  a  particular  measurement  interval  under  stated  conditions. 

Mean  Time  Between  Removals  (MTBR)  -  Total  number  of  system  life  units  (i.e.,  op¬ 
erating  hours,  flight  hours,  etc)  divided  by  the  number  of  removals. 

MODAS  -  The  Maintenance  and  Operational  Data  Access  System  (MODAS)  is  an  inter¬ 
active  Database  Management  System  (DBMS)  containing  24  months  of  field  and  depot 
reported  operational  (inventory,  status,  and  utilization)  and  maintenance  (AFR  6G-1) 
data.  The  system  is  being  developed  primarily  to  support  ALC  System  Program  Man¬ 
agers  (SPMs)  and  Inventory  Managers  (IMs).  The  database  contains  summarized 
files  for  obtaining  Reliability  and  Maintainability  information  and  detailed  data  files 
such  as  non-mission  capable  hours  per  flight  hour,  sorties,  or  landings  by  Command 
or  Base. 

Navy  3M  Data  System  -  Navy  Maintenance  Material  Management  System,  output  pro¬ 
cess  data  tapes  and  reports  compiled  in  accordance  with  Navy  Aviation  Maintenance 
Program  (NAMP-4790). 

Off  Equipment  -  Maintenance  Actions  that  occur  away  from  (off)  end  item  article, 
e.g.,  intermediate  repair  shop. 

On  Equipment  -  Maintenance  Actions  that  are  exhibited  at  (on)  end-item  article, 
e.g.,  weapon  system. 

Percent  Defective  (d)  -  The  number  of  defective  units,  divided  by  the  number  of 
units  of  product. 

Process  Average  (y)  -  The  average  number  of  defects  per  unit  per  specified  interval 
of  time  (expressed  in  terms  of  removals  in  this  guideline). 


Removal  Rate  -  Reciprocal  of  MTBR. 


Removals  -  The  number  of  items  removed  from  a  system  during  a  stated  period  of 
time  as  related  to  demand  for  logistic  support,  and  excluding  removals  performed  to 
facilitate  other  maintenance,  removals  for  product  improvement,  and  removals  for 
cannibalization . 

Repairable  -  An  item  that  can  be  restored  to  perform  all  its  required  functions  by 
corrective  maintenance. 

Service  Life  -  The  duration  of  time  an  item  experiences  in  operational  inventory,  in¬ 
cluding  the  performance  of  any  maintenance  act  to  keep  the  item  in  operating  condi¬ 
tion  . 


Shop  Replaceable  Unit  (SRU)  -  An  assembly  or  any  combination  of  parts,  subassem¬ 
blies,  and  assemblies  mounted  together,  normally  capable  of  independent  operation  in 
a  variety  of  situations  and  repairable  at  an  ALC. 

Specified  MTBF(s)  -  Design  or  operational  objective,  as  defined  by  handbook  pre¬ 
dicting  techniques  (i.e.,  MIL-HDBK-217) ,  specified  contractual  goals  (i.e.,  war¬ 
ranties),  field  operational  objectives  (i.e.,  R&M  2000  targets),  or  logistic  planners, 
goals  (i.e.,  wartime  loading  levels). 

System  -  A  group  of  interconnected  electronic  units  which  provides  a  specific  func¬ 
tion  (e.g.,  radar  system,  navigation  system,  etc). 

Tailoring  -  A  process  of  environmental  surveys  required  when  the  generic  ESS 
environmental  levels  exceed  the  unit's  functional  design  qualification  levels.  The 
potential  exceedance  levels  are  reduced  or  notched  at  resonant  frequencies  to 
eliminate  structural  and  intermittant  electrical  problems. 

Work  Unit  Code  (WUC)  -  An  alphanumeric  code  assigned  to  individual  systems,  sub¬ 
systems,  and  equipment  within  a  weapon  system  (aircraft,  ground  system,  missile, 
etc)  to  track  maintenance  activities. 
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10.2  ACRONYMS  &  ABBREVIATIONS 


AGREE 
AFC  - 
AFLC  - 
ALC  - 
AT  - 
ATE  - 
CND  - 
DR  - 
DTC  - 
ESS  - 
HM  - 
ICS  - 
JCN  - 
LRU  - 
LSC  - 
LSCS  - 
MTBF  - 
MTBR  - 
NOC  - 
PCB  - 
ROI  - 
SRD  - 
SRU  - 
TM 

WUC  - 


-  Advisory  Group  on  Reliability  of  Electronic  Equipment 
Amortized  Facility  Cost 
Air  Force  Logistics  Command 
Air  Force  Logistics  Center 
Action  Taken 
Automatic  Test  Equipment 
Cannot  Duplicate 
Discrimination  Ratio 
Direct  Test  Cost 
Environmental  Stress  Screening 
How  Malfunction 
Integrated  Circuits 
Job  Control  Number 
Line  Replaceable  Unit 
Logistic  Support  Cost 
Logistic  Support  Cost  Savings 
Mean  Time  Between  Failure 
Mean  Time  Between  Removal 
Not  Otherwise  Coded 
Printed  Circuit  Board 
Return  on  Investment 
Standard  Reporting  Designation 
Shop  Replaceable  Unit 
TM-Type  Maintenance 
Work  Unit  Code 


APPENDIX  A  -  LOGISTIC  SUPPORT  COST  MODEL 


The  model  used  to  determine:  downstream  logistic  support  costs  and  cost  savings 
resulting  from  application  of  ESS  to  an  LRU/SRU  was  the  USAF  Logistic  Support 
Cost  (L3C)  Model,  Version  1.1.  This  widely  used  model  developed  for  avionics 
systems  uses  algorithms  and  accounting  equations  which  are  documented  in  the  AFLC 
User's  Handbook.  The  model  output  included  the  following  elements  of  logistic 
support: 

•  LRU/SRU  spares 

•  On-equipment  maintenance 

•  Off-equipment  maintenance 

•  Inventory  management 

•  Support  equipment 

•  Training 

•  Management  and  technical  data. 

The  model  used  approximately  52  input  variables  describing  the  system  and 
maintenance  scenario,  and  approximately  25  input  variables  describing  each  LRU. 
This  input  data  was  obtained  from  sources  such  as  the  manufacturer  of  the  LRU,  Air 
Force  AFM  66- 1  maintenance  data,  Air  Force  AFLCP  ’73-10,  "AFLC  Cost  and  Plan¬ 
ning  Factors,"  and  AF  Regulation  173-13  "USAF  Cost  and  Planning  Factors." 

Tables  A-l  and  A-2  show  the  list's  weapon  system  and  system  input  variables 
used  in  the  model  for  the  three  aircraft.  Table  A- 3  shows  the  LRU  input  variables 
for  the  E-2C  and  F-14  avionic  boxes.  Table  A- 4  shows  the  LRU  input  variables  for 
the  EF-111A  avionic  boxes. 


TABLE  A-1.  List  of  weapon  system  variables. 


1.  IMC 


2.  M 


MRF 


MRO 


NFLUSW 


NSESW 


NSVS 
8.  I  OS 


9.  O ST CON 


OSTOS 


t2. 


13. 


14. 


16. 


PSC 


PSO 


RMC 


17. 


18. 


19. 


20. 


SA 


SR 


TARGAVAL 


TO 


21. 


22 


23 


26. 


TFFH 


TR 


TR0 


|TR0 


UEBASE 


INITIAL  MANAGEMENT  COST  TO  INTRODUCE  NEW  LINE  ITEM 
OF  SUPPLY  (ASSEMBLY  OR  PIECE  PART)  INTO  AIR  FORCE 
INVENTORY  (S)  (AFLCR  173-10) _ 

NO.  INTERMEDIATE  REPAIR  LOCATIONS  (OPERATING  BASES) 
IP)  (SEE  NOTE  2) 


AVG  MANOURS  PER  FAILURE  TO  COMPLETE  OFF -EQUIPMENT 
MAINTENANCE  RECORDS  (S  -  0.24  HOURS) 


avg  Manhours  per  failure  to  complete  on-eouipment 

MAINTENANCE  RECORDS  (S  -  OjOE'  HOURS) 


NO.  FLU  SCFTY.AR0  PACKAGES  W.TKIN  WEAPON  SYSTEM 
1C)  (ASSUMED) 


NO.  SE  SOFTWARE  PACKAGES  WITHIN  WEAPON  SYSTEM  (C) 
(ASSUMED) _ 


NO.  SYSTEMS  WITHIN  WEAPON  SYSTEM  (C)  {NSVS  -  1) 
FRACTION  OF  TOTAu  FORCE  DEPLOYED  TO  OVERSEAS 

LOCATIONS  (P)  (SEE  NOTE  2) _ 

WEIGHTED  AVERAGE  ORDER  A  SHIPPING  TIME  IN  MONTHS. 
ELAPSED  TIME  BETWEEN  INITIATION  OF  REQUEST  FOR  A 
SERVICEABLE  ITEM  A  ITS  RECEIPT  BY  REQUESTING 
ACTIVITY.  FOR  CONUS  LOCATIONS  LOCATIONS.  S  -  0394 
MONTHS  (12  DAYS)  INPUT  AS  OSTCDN 


SAME  ASOSTCON  EXCEPT  FOR  OVERSEAS  LOCATIONS, 

S  -  0.526  MONTHS  (18  DAYS)  INPUT  AS  OSTOS.  (AFLCR 

173-10):  OST  -  (OSTCON)  (1-OS)  +  (OSTQS)  (OS) _ 

PEAK  FORCE  FLYING  HOURS  -  EXPECTED  FLEET  FLYING 
HOURS  FOR  ONE  MONTH  DURING  PEAK  USAGE  PERIOD  (P) 

(SEE  NOTE  3) _ 

OPERATIONAL  SERVICE  LIFE  OF  WEAPON  SYSTEM  IN  YEARS 
(PROGRAM  INVENTORY  USAGE  PERIOD)  (ASSUMED  PIUP  - 
15  YEARS) 


OIRECT  PRODUCTIVE  MANHOURS  PER  MAN  PER  YEAR  AT 
BASE  LEVEL  (INCLUOES  "TOUCH  TIME,"  TRANSPORTATION 
TIME.  AND  SETUP  TIME)  (S  -  1728  HOURS/M  AN/YEAR) 
(AFLCR  173-10)  _ 


DIRECT  PRODUCTIVE  MANHOURS  PEF.  MAN  PER  YEAR  AT 
DEPOT  (INCLUOES  'TOUCH  TIME."  TRANSPORTATION 
TIME.  AND  SETUP  TIME)  (S  -  1728  HOURS/M AN^T EAR) 

(AFLCR  173-10) _ 

AVG  PACKING  AND  SHIPPING  COST  TO  CONUS  LOCATIONS 
(S)  (AFLCR  173-10)  _  _ 


AVG  PACKING  AND  SHIPPING  COST  TO  OVERSEAS 

LOCATIONS  (S)  (AFLCR  1 73-10) _ 

RECURRING  MANAGEMENT  COST  TO  MAINTAIN  A  L'NE  ITEM 
OF  SUPPLY  (ASSEMBLY  OR  PIECE  PART)  IN  WHOLESALE 

INVENTORY  SYSTEM  (S)  (AFLCR  173-10)  _ 

ANNUAL  8ASE  SUPPLY  LINE  ITEM  INVENTORY  MANAGEMENT 

COST  (S)  (AFLCR  173-10) _ 

AVG  MANHOURS  TO  COMPLETE  SUPPLY  TRANSACTION 

RECORDS  (S) _ 

BASE  LEVEL  SPARES  AVAILABILITY  OBJECTIVE  FOR  WEAPON 
SYSTEM  (H  (ASSUMED  TARGAVAL -0  95)  _ 


AVG  COST  PER  ORIGINAL  PAGE  OF  TECHNICAL  DOCUMEN¬ 
TATION.  AVERAGE  ACQUISITION  COST  OF  ONE  PAGE  OF 
THE  REPRODUCIBLE  SOURCE  DOCUMENT  (DOES  NOT 
INCLUDE  REPRODUCTION  COSTS)  (S)  (AFLCR  173-10)  _ 


EXPECTED  TOTAL  FORCE  FLY  ING  HOURS  OVER  PR'  CRAM 

INVENTORY  USAGE  PERIOD  (P)  (SEE  NOTE  4) _ 

AVG  MANHOURS  PER  FAILURE  TO  COM°LETE  TRANSPOR¬ 
TATION  TRANSACTION  FORMS  <S  •  .16  HOURS)  _ 


ANNUAL  TURNOVER  RATE  FOR  B/SE  PERSONNEL 

(S-  .134) _ _ 

ANNUAL  TURNOVER  RATE  FOR  DEPOT  PERSONNEL 

(S  -  .15) _ _ 

NO.  UNIT  EQUIVALENT  WEAPON  SYSTEMS  PER  OPERATING 
BASE  (P) 


0J4 


a  .S3 


1728 


1728 


2.1B 


03* 


1 203200 


0.13 


0.16 


243  "9 


3360. 


1728 


449280 


0.16 


0.13 


243.89 


0/38 


039 


0.53 


26 


1722 


2.10 


1231 


306.51 


0.16 


0.13 


12. 


R87*3772*044  (T) 


A-2 


r 


■  —  .viim w-d tA*vn. *•„ >r,.Yvyv*i*YVY,»'ir.,^^^^y*^^_v. "><  ■>  v  v 


TABLE  A-2.  List  of  system  variables  (sheet  1  of  2). 


F-14 

E-2C 

EF-111A 

1 . 

BCA 

TOTAL  COST  OF  ADDITIONAL  ITEMS  OF  COMMON  BASE  SHOP  SUPPORT 
EQUIPMENT  PER  BASE  REQUIRED  FOR  SYSTEM  (CS  IBCA  -  01 

0 

0 

0 

2. 

BAA 

AVAILABLE  WORK  TIME  PER  MAN  IN  BASE  SHOP  IN  MAN  HOURS  PER 
MONTH  (S  -  168  HOURS)  (AFLCR  173-10) 

168 

168 

168 

3. 

BLF* 

BASE  LABOR  RATE,  INCLUDING  INDIRECT  LABOR,  INDIRECT 

MATERIAL  &  OVERHEAD  IS)  (AFLCR  173-10) 

22.27 

22.27 

2227 

4. 

BMR 

BASE  CONSUMABLE  MATERIAL  CONSUMPTION  RATE.  INCLUDES 

MINOR  ITEMS  OF  SUPPLY  (NUTS,  WASHERS,  RAGES,  CLEANING  FLUID, 
ETC)  CONSUMED  DURING  REPAIR  OF  ITEMS  (SI  IAFLCR  173*10) 

3.23 

3.23 

3.23 

5. 

BPA 

TOTAL  COST  OF  PECULIAR  BASE  SHOP  SUPPORT  EQUIPMENT  PER 

BASE  REQUIRED  FOR  SYSTEM  WHICH  IS  NOT  DIRECTLY  RELATED  TO 
SPECIFIC  PLUS  OR  WHEN  QUANTITY  REQUIRED  IS  INDEPENDENT  OF 
ANTICIPATED  WORKLOAD  (SUCH  AS  OVERHEAD  CRANES  AND  SHOP 
FIXTURES)  (C)  (BPA  >0) 

0 

0 

0 

6. 

BRCT 

AVj  BASE  REPAIR  CYCLE  TIME  IN  MONTHS.  THE  ELAPSED  TIME  FOR 

AN  RTS  ITEM  FROM  REMOVAL  OF  FAILED  ITEM  UNTIL  IT  IS  RE¬ 
TURNED  TO  BASE  SERVICEABLE  STOCK  (LESS  TIME  AWAITING 

PARTS).  FOR  FLUS  OF  THE  ''BLACK  BOX"  VARIETY  (E.G..  AVOINIOS 
LRUs),  THE  REPAIR  OF  WHICH  NORMALLY  CONSISTS  OF  REMOVAL  & 
REPLACEMENT  OF  "PLUG-IN"  COMPONENTS  (SRUs),  S  -  0.13  MONTHS 
(4  DAYS).  (FOR  OTHER  NONMOOULAR  FLUs,  S  -0.20  MONTHS  (6 

DAYS)  (AFLCR  173-10)  (BRCT  -  0.13) 

0.13 

0.13 

7. 

cs 

COST  OF  SOFTWARE  TO  UTILIZE  EXISTING  AUTOMATIC  TEST 
EQUIPMENT  FOR  SYSTEM  (C)  (CS  -  0) 

0 

0 

0 

8. 

DCA 

TOTAL  COST  OF  ADDITIONAL  ITEMS  OF  COMMON  DEPOT  SUPPORT 
EQUIPMENT  REQUIRED  FOR  SYSTEM  (C)  (DCA  -  0) 

0 

0 

0 

9. 

□AA 

AVAILABLE  WOPK  TIME  TER  MAN  AT  DEPOT  IN  MANHOURS  PER 

MONTH  (S  *  168  HOURS)  (AFLCR  173-10) 

168 

168 

168 

10. 

OLR 

DEPOT  LABOR  RATE,  INCLUDING  OTHER  DIRECT  COSTS.  OVER¬ 
HEAD  &  G4A  (S)  (AFLCR  173-10) 

38.44 

38.44 

38.44 

11. 

DMR 

SAME  AS  BMR  EXCEPT  REFERS  TO  DEPOT  LEVEL  MAINTENANCE  (S) 
(AFLCR  173-10) 

11.78 

11.78 

11.78 

12. 

DPA 

SAME  AS  BPA  EXCEPT  RELATED  TO  DEPOT  SUPPORT  EQUIPMENT 
(C)  (DPA  -  0) 

0 

0 

0 

13. 

DRCT 

WEIGHTED  AVERAGE  DEPOT  REPAIR  CYCLE  TIME  IN  MONTHS.  THE 
ELAPSED  TIME  FOR  THE  NRTS  ITEM  FROM  REMOVAL  OF  FAILED 

ITEM  UNTIL  IT  IS  RETURNED  TO  DEPOT  SERVICEABLE  STOCK. 

INCLUDES  TIME  REQUIRED  FOR  BASE-TO-DEPOT  TRANSPORTATION 
ft  HANDLING  a  SHOP  FLOW  TIME  WITHIN  SPECIALIZED  REPAIR 

ACTIVITY  REQUIRED  TO  REPAIR  ITEM.  FOR  CONUS  LOCATIONS, 

S  -  1.73  MONTHS  (52  DAYS)  FOR  ORGANIC  REPAIR,  S  -  2.06  MONTHS 
(62  DAYS)  FOR  CONTRACTI  ’  *  L  REPAIR.  INPUT  AS  DRCTC.  FOR 
OVERSEAS  LOCATIONS,  L  -  1.90  MONTHS  (57  DAYS)  FOR  ORGANIC 
REPAIR.  S  -  2.20  MONTHS  (66  DAYS)  FOR  CONTRACTUAL  REPAIR,  IN¬ 
PUT  AS  DRCTO  (AFLCR  173-10)  DRCT -(DRCT)  (1 -OS)  +  (DRCTO)  (OS) 

(SEE  NOTE  2) 

1.73 

1.73 

’ 

1.73 

14. 

FB 

TOTAL  COST  OF  NEW  8ASE  FACILITIES  CNCLUDING  UTILITIES)  TO 

BE  CONSTRUCTED  FOR  OPERATION  a  MAINTENANCE  OF  SYSTEM,  IN 
DOLLARS  PER  BASE  (C)  (FB  -  0) 

■ 

■ 

° 

15. 

FD 

TOTAL  COST  OF  NEW  DEPOT  FACILITIES  (INCLUDING  UTILITIES),  Tu 

BE  CONSTRUCTED  FOR  MAINTENANCE  OF  THE  SYSTEM)  (C)  (FD  -  0) 

0 

0 

0 

16. 

FLA 

TOTAL  COST  OF  PECULIAR  FLIGHT-LINE  SUPPORT  EQUIPMENT  a 
ADDITIONAL  'TEMS  OF  COMMON  FLIGHT-LINE  SUPPORT  EQUIPMENT 

PER  BASE  REQUIRED  FOR  SYSTEM  (C)  (ASSUMED) 

0 

0 

0 
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TABLE  A-2.  List  of  system  variables  (sheet  2  of  2). 


E-2C  EF-111A 


H 

NO.  PAGES  OF  OEPOT  LEVEL  TECHNICAL  ORDERS  &  SPECIAL  RE¬ 
PAIR  INSTRUCTIONS  REQUIRED  TO  MAINTAIN  SYSTEM  (C)  (ASSUMED) 

IH 

COST  OF  INTERCONNECTING  HARDWARE  TO  UTILIZE  EXISTING 
AUTOMATIC  TEST  EQUIPMENT  FOR  SYSTEM  (C)  (IH  -  0) 

JJ 

NO.  PAGES  OF  ORGANIZATIONAL  &  INTERMEDIATE  LEVEL  TECHNI¬ 
CAL  ORDERS  REQUIRED  TO  MAINTAIN  SYSTEM  (C)  (ASSUMED) 

N 

NUMBER  OF  DIFFERENT  F  LUs  WITHIN  SYSTEM  (C) 

SMH 

AVG  MANHOURS  TO  PERFORM  A  SCHEDULED  PERIODIC  OR  PHASED 
INSPECTION  ON  SYSTEM  (C)  (SMH  -  0) 

SMI 

FLYING  HOUR  INTERVAL  BETWEEN  SCHEDULED  PERIODIC  OR 

PHASED  INSPECTIONS  ON  SYSTEM  (C)  (SEE  NOTE  4) 

SYSNOUN 

NAME  OF  SYSTEM  -  UP  TO  60  ALPHANUMERIC  CHARACTERS  (C) 

TCB 

COST  OF  PECULIAR  TRAINING  PER  MAN  AT  BASE  LEVEL  IN¬ 
CLUDING  INSTRUCTION  &  TRAINING  MATERIALS  (C)  (SEE  NOTE  6) 

TCD 

COST  OF  PECULIAR  TRAINING  PER  MAN  AT  DEPOT  INCLUDING 
INSTRUCTION  &  TRAINING  MATERIALS  (C)  (SEE  NOTE  6) 

TE 

COST  OF  PECULIAR  TRAINING  EQUIPMF.VT  REQUIRED  FOR 

SYSTEM  (C)  (SEE  NOTE  6) 

XSYS 

SYSTEM  IDENTIFICATION.  ASSIGNED  FIVE-CHARACTER  ALPHANU¬ 
MERIC  WORK  UNIT  CODE  OF  SYSTEM  (Cl 

prpvwwr  wren  vrirn  ir*ar  rm  laiumoiumswvrM  ■ 


TABLE  A*3.  List  of  LRU  variables  (sheet  1  of  2). 


LINE  REPLACEABLE  UNITS-DEFINIT10NS,  SOURCES  ft  RATIONALE  |  VARIABLE  | 


AVG  MANHOURS  TO  PERFORM  SHOP  BENCH  CHECK.  SCREENING 
&  FAULT  VERIFICATION  ON  REMOVED  FLU  PRIOR  TO  INITIATING 
REPAIR  ACTION  OR  CONDEMNING  THE  ITEM  (C)  13  M  DATA) 


AVG  COST  PER  FAILURE  FOR  FLU  REPAIRED  AT  BASE  LEVEL 
FOR  STOCKAGE  &  REPAIR  OF  LOWER  LEVEL  ASSEMBLIES 
EXPRESSED  AS  A  FRACTION  OF  FLU  UNIT  COST  (UC).  THIS  IS  THE 
IMPLICIT  REPAIR,  DISPOSITION  COST  FOR  A  FLU  REPRESENTING 
LABOR,  MATERIAL  CONSUMPTION.  ft  STOCKAGl  REPLACEMENT 
OF  LOWER  INDENTURE  REPAIRABLE  COMPONENTS  WITHIN  THE 
FLU  (c.g.,  SHOP  REPLACEABLE  UNITS  OR  MODULES  (0 
(SEE  NOTE  6) 


AVG  MANHOURS  TO  PERFORM  INTERMEOI ATE-LEVEL  (BASE 
SHOP)  MAINTENANCE  ON  REMOVED  FLU  INCLUDING  FAULT 
ISOLATION.  REPAIR,  &  VERIFICATION.  (C)  (3-M  DATA) 


FRACTION  OF  REMOVED  FLU*  EXPECTED  TO  RESULT  IN 
CONDEMNATION  AT  BASE  LEVEL  (C)  (ASSUMED) 


SAME  AS  BBCMH  EXCEPT  REFERS  TO  DEPOT  LEVEL 
MAINTENANCE  (0  (3  M  OATA) 


FRACTION  OF  FLU*  RETURNED  TO  DEPOT  FOR  REPAIR  (NRTS) 
EXPECTED  TO  RESULT  IN  CONDEMNATION  OF  DEPOT  LEVEL 
(C)  (ASSUMED) 


SAME  AS  BMC  ESCEPT  REFERS  TO  DEPOT  REPAIR  ACTIONS 
(C)  (SEE  NOTE  6) 


SAME  AS  BMH  EXCEPT  REFERS  TO  DEPOT  LEVEL  MAINTENANCE 
(C)  (3-M  OATA) 


WORD  DESCRIPTION  OR  NAME  OF  THE  FLU  -  UP  TO  60 
ALPHANUMERIC  CHARACTERS  (C) 


AVG  MANHOURS  TO  PERFORM  CORRECTIVE  MAINTENANCE 
OF  FLU  IN  PLACE  OR  INLINE  WITHOUT  REMOVAL  INCIUOING 
FAULT  ISOLATION.  REPAIR  ft  VERIFICAT‘ON  (C)  (3-M  OATA) 


NO.  LINE  ITEMS  OF  PECULIAR  SHOP  SUPPORT  EQUIPMENT  USEO 
IN  REPAIR  OF  THE  FLU  (0 


AVG  MEAN  TIME  BETWEEN  FAILURES  IN  OPERATING  HOURS 
OF  FLU  IN  OPERATION  ENVIRONMENT  WITHOUT  ESS  (C) 

(3-M  OATA) 


SAME  AS  12  EXCEPT  ESS  INITIALLY  APPLIED  TO  EACH  LRU  (0 
(3-M  DATA) 


FRACTION  OF  REMOVEO  FLU*  EXPECTEO  TO  BE  RETURNED  TO 
THE  DEPOT  FOR  REPAIR  (C)  (SEE  NOTE  5) 


NEW  T  COOEO  REPAIRABLE  ASSEMBLIES  WITHIN  THE  FLU  (0 
(SEE  NOTE  6) 


AVG  MANHOURS  EXPENDEO  IN  PLACE  ON  INSTALLED  SYSTEM 
FOR  PREPARATION  ftACCESS  VFOR  FLU;  FR  EXAMPLE.  JACK¬ 
ING.  UNBUTTONING.  REMOVAL  OF  OTHER  UNITS  AND  HOOKUP 
OF  SUPORT  EQUIPMENT  (C)  (3-M  OATA) 


NO.  NEWT*  COOEO  CONSUMABLE  ITEMS  WITHIN  THE  FLU  (C) 
(SEE  NOTE  6) 


QUANTITY  OF  LIKE  FLUl  WITHIN  PARENT  SYSTEM  (QUANTITY 
PER  APPLICATIONS  (0 


SiGNAL  COMMANO  COMPUTER  CENTRAL  AIR  VERTICAL  AIR  INLET 

READOUT  ALARM  SIGNAL  OATA  OATA  OISPLAY  CONTROL 

MODULE  CONVERTER  COMPUTER  GROUP  SYSTEM 

GROUP 
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TABLE  A-3.  List  of  LRU  variable*  (sheet  2  of  2). 


i 


i 


LIME  REPLACEABLE  UNITS-OEFINITIONS.  SOURCES  •  RATIONALE 

AIRCRAFT 

iir 

VARIABLE 

SCRA'J 

csoc 

13. 

FRACTION  OF  FLU  FAILURES  WHICH  CAN  BE  RcPAIREO  IN  FLACE 
OR  ON  LINE  WITHOUT  REMOVAL  ICI  (3  M  OATAI 

RIP 

123 

113 

11 

AVG  MANHOURS  TO  FAULT  ISOLATE.  REMOVE.  ANO  REPLACE 

FLU  ON  INSTALLEO  SYSTEM  A  VERIFY  RESTORATION  OF 

SYSTEM  TO  OPERATIONAL  STATUS  (Cl  (3  M  OATAI 

1 

| 

11 

2a 

FRACTION  OF  REMOVED  FLUl  EXPECTED  TO  BE  REPAIRED  AT 
BASE  LEVEL  (Cl  (Ml  OATAI 

RTS 

0.99 

IS* 

21. 

NO.  STANDARD  (ALREADY  STOCX  NUMSEREOI  PARTS  WITHIN 

FLU  WHICH  WlllL  BE  MANAGED  FOR  FIRST  TIME  AT  BASES 

WHERE  THIS  SYSTEM  IS  OEPLOYEO  (Cl  (SEE  NOTE  SI 

SP 

0 

ft 

22. 

EXPECTED  UNIT  COST  OF  THE  FLU  AT  TIME  OF  INITIAL 
PROVISIONING  (C)  (ESTIMATES  BY  SUPPLY  OPERATIONS 
ANALYSTS! 

uc 

55217 

312000 

21 

RATIO  OF  OPERATING  HOURS  TO  FLYING  HOURS  FOR  FLU 
(USE  FACTOR)  (Cl  (UF  -  1.2*  -  ASSUMED) 

UF 

12* 

US 

H. 

FLU  UNIT  WEIGHT  IN  POUNOS  (Cl  (DESIGN  SPECIFICATIONS) 

w 

10 

40 

21 

FLU  IDENTIFICATION.  THE  ASSIGNEO  FIVE-CHARACTER 
ALPHANUMERIC  WORK  UNIT  CODE  OF  THE  FLU  (C)  (USED 

WUC  WHEN  AVAILABLE) 

XFLU 

51211 

51X44 
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TABLE  A-4.  List  of  LRU  variables  —  2. 


LIME  REPLACEABLE  UmT^OEFlNITIQNS.  SOURCES,  *  RATIONALE 


AIRCRAFT 

VARIABLE 


1.  AVG  MANHOURSTO  PERFORM  SHOP  BENCH  CHECK.  SCREENING.  &  FAULT  BBCMH 
VERIFICATION  ON  REMQVEO  HU  PRIOR  TO  INITIATING  REPAIR  ACTION 

OR  CONDEMNING  ITEM  (C)  (3MOATAI 

2.  AVC  COST  PER  FAILURE  FOR  FUJ  REPAIRED  AT  BASE  LEVEL  FOR  STOCK 
AGE  A  REPAIR  OF  LOWER  LEVEL  ASSEM8UES  EXPRESSED  AS  A  FRAC 
TION  OF  FLU  UNIT  COST  IUCK  THIS  IS  THE  IMPLICIT  REPAIR  DISPOSITION 
COST  FOR  Flu  representing  labor,  material  CONSUMPTION,  a 
STOCK  AGE/ RE  PLACEMENT  OF  ■  OWER  INDENTURE  COMPONENTS  WITHIN 
THE  FLU  !E  G  .  SHOP  REPLACEABLE  UNITS  OR  MODULES  ICI 
(SEE  NOTE  6) 


3.  AVG  MANHOURSTO  PERFORM  INTERMEDIATE  LEVEL 'BASE  SHOPI  j  BMH 

MAINTENANCE  ON  A  REMOVED  FLU  INCLUDING  FAULT  ISOLATION  RE  I 
PAIR  A  VERIFICATION  (CM3  M  OATA) 


.  |  FRACTION  OF  REMOVED  FLU*  EXPECTED  TO  RESULT  IN  CONOEMNA 
j  TION  AT  BASE  LEVEL  (C)  (ASSUMED! 


.  SAME  AS  BBCMH  EXCEPT  REFERS  TO  DEPOT  LEVEL  MAINTENANCE  ICI  DBCMH 
(3-M  OATA) 


FRACTION  OF  FLU*  RETURNED  TO  DEPOT  FOR  REPAIR  (NRTSI  EX  QCONO 

PECTED  TO  RESULT  IN  CONDEMNATION  OF  OEPOT  LEVEL  (Cl  IASSUMEO) 


1.  |  SAME  AS  BMC  EXCEPT  REFERS  TO  OEPOT  REPAIR  ACTIONS  (Cl 
(SEE  NOTE  61 


SAME  AS  BMH  EXCEPT  REFERS  TO  OEPOT  LEVEL  MAINTENANCE  ICI 
(3-M  OATA) 


WORD  DESCRIPTION  OR  NAME  OF  THE  FLU  -  UP  TO  60  ALPHANU 
MERIC  CHARACTERS  (0 


AVG  MANHOURS  TO  PERFORM  CORRECTIVE  MAINTENANCE  OF  FLU  IN 
PLACE  OR  IN  LINE  WITHOUT  REMOVAL  INCLUDING  FAULT  ISOLATION. 
REPAIR  &  VERIFICATION  (C)  (3-M  DATA! 


NO.  LINE  ITEMS  OF  PECULIAR  SHOP  SUPPORT  EQUIPMENT  USEO  IN 
REPAIR  OF  FLU  (Cl 


AVG  MEAN  TIME  BETWEEN  FAILURES  IN  OPERATING  HOURS  OF  THE  FLU 
IN  OPERATION  ENVIRONMENT  WITHOUT  ES3  id  (3-M  OATA) 


12A.  SAME  AS  12  EXCEPT  ESS  INITIALLY  APPLIED  TO  EACH  LRU  (Cl  (3  M  DATA) 


FRACTION  OF  REMOVED  FLU*  EXPECTED  TO  BE  RITURNEO  TO  OEPOT 
FOR  REPAIR  (Cl  (SEE  NOTE  5) 


NO.  OF  NEW  "P"  COOEO  REPAIRABLE  ASSEMBLIES  WITHIN  THE  FLU  (C) 
(SEE  NOTE  6) 


IS.  AVG  MANHOURS  EXPENDED  IN  PLACE  ON  INSTALLED  SYSTEM  FOR 

PREPARATION  A  ACCESS  FOR  FLU;  FOR  EXAMPLE.  JACKING.  UNBUTTON¬ 
ING.  REMOVAL  OF  OTHER  UNITS  &  HOOKUP  OF  SUPPORT  EQUIPMENT  (C) 
(3  M  DATA) 


NO.  NEW  •' r  COOEO  CONSUMABLE  ITEMS  WITHIN  FLU  (C)  (SEE  NOTE  6) 


QUANTITY  OF  LIKE  FLUtWITHIN  PARENT  SYSTEM  (QUANTITY  PER 
APPLICATIONS)  (Cl 


FRACTION  OF  FLU  FAILURES  WHICH  CAN  BE  REPAIRED  IN  PLACE  OR  ON 
LINE  WITHOUT  REMOVAL  (C)  (3-M  OATA) 


AVG  MANHOURS  TO  FAULT  ISOLATE.  REMOVE.  ANO  REPLACE  FLU  ON 
INSTALLED  SYSTEM  ANO  VERIFY  RESTORATION  OF  SYSTEM  TO  OPERA¬ 
TION  STATUS  ICI  13-M  OATA) 


FRACTION  OF  REMOVED  FLU*  EXPECTEO  TO  BE  REPAIRED  AT  BASE 
LEVEL  (C)  13  M  OATA) 


NO.  STANDARO  (ALREADY  STOCK  NUMBEREO)  PARTS  WITHIN  FLU  WHICH 
WILL  BE  MANAGED  FOR  FIRST  TIME  AT  BASES  WHERE  THIS  SYSTEM  IS 
DEPLOYED  (C)  (SEE  NOTE  S) 


EXPECTED  UNIT  COST  OF  FLU  AT  TIME  OF  INITIAL  PROVISIONING  (C) 
(ESTIMATES  BY  SUPPLY  OPERATIONS  ANALYSIS) 


RATIO  OF  OPERATING  HOURS  TO  FLYING  HOURS  FOR  FLU  (USE  FACTOR) 
10  (UF-I  ASSUMED) 


24.  (FLU  UNIT  WEIGHT  IN  POUNDS  (C)  (DESIGN  SPECIFICATIONS) 


11 

11 

IB 

11 

11 

401300 

3024» 

4442S0 

4ISB00 

173700 

1J2S  US  US  US  US 


NOTES  TO  LIST  OF  WEAPON,  SYSTEM  &  LRU  VARIABLES 


Note  1 


Note  2 


Note  3 


Note  4 


:  (C)  =  Contractor- furnished  value 
(S)  =  Government- furnished  standard  value 
(P)  =  Program-peculiar  value 

(AFLCR  173-10)  =  denotes  data  source  is  AFLCR  173-10  "AFLC  Cost  and 
Planning  Factors" 


:  Assumption: 

A  total  of  360  F-14  aircraft  will  be  deployed  with  60  each  at  Miramar  and 
Oceana  and  on  each  of  four  aircraft  carriers.  Therefore,  OS  =  0.66  for 
the  F-14. 

A  total  of  48  E-2C  aircraft  will  be  deployed  with  16  each  at  Miramar  and 
Oceana  and  four  each  aboard  four  aircraft  carriers.  Therefore,  OS  = 
0.33  for  the  E-2C. 

A  total  of  36  EF-111A  aircraft  will  be  deployed  with  12  each  at  three 
bases,  one  base  being  overseas.  Therefore,  OS  =  0.33  for  the  EF-111A. 

:  Assumption: 

F-14  Peak  Flying  Hours  =  50  hours  per  aircraft  per  month.  Therefore 
Peak  Force  Flying  Hours  =  50  hours  x  360  aircraft  =  18,000  hours. 

E-2C  Peak  Flying  Hours  =  70  hours  per  aircraft  per  month.  Therefore 
Peak  Force  Flying  Hours  =  70  hours  x  48  aircraft  =  3660  hours. 

EF-111A  Peak  Flying  Hours  =  50  hours  per  aircraft  per  month.  Therefore 
Peak  Force  Flying  Hours  =  50  hours  x  36  aircraft  =  1800  hours. 

:  Assumption: 

F-14  Average  Flying  Hours  =  34  hours  per  aircraft  per  month  and  opera¬ 
tional  service  life  =  15  years.  Therefore  Total  Force  Flying  Hours  =  34 
hours  x  12  months  x  15  years  x  360  aircraft  =  2,203,200. 


E-2C  Average  Flying  Hours  =  52  hours  per  aircraft  per  month  and  the 
operational  service  life  =  15  years.  Therefore  Total  Force  Flying  Hours  = 
52  hours  x  12  months  x  15  years  x  48  aircraft  =  449,280  hours. 

EF-111A  Average  Flying  Hours  =  30  hours  per  aircraft  per  month  and  the 
operational  service  life  =  15  years.  Therefore  Total  Force  Flying  Hours  = 
30  hours  x  12  months  x  15  years  x  3G  aircraft  =  194,400  hours.  For  an 
operational  service  life  of  25  years,  the  TFFH  =  324,000  hours. 

Note  5:  The  LRUs  on  the  F-14,  E-2C,  and  EF-111A  were  assumed  to  have  no  sched¬ 
uled  periodic  or  phased  inspections.  Therefore,  the  interval  between  them 
was  infinite  so  that  a  value  of  999,999,999  was  used. 

Note  6:  BMC  and  DMC  =  0.02  is  used  for  all  LRUs.  The  value  is  based  on  data  of 
the  F-16  "Logistic  Support  Cost  Status  Report  (UL  76AQ),"  15  June  1981  for 
similar  equipment. 

Note  7:  Average  numbers  for  PA,  PP,  and  SP  were  estimated  by  our  Supply  Opera¬ 
tions  analysts. 


APPENDIX  B  -  HIGH  RANK  REMOVAL  LRUs 
EF-111A  (June  1984  -  July  1986) 


Rank 

wuc 

Removals** 

MFHBR* 

1 

76ZMO 

662 

31 

2 

76ZKO 

616 

33 

3 

76ZGO 

547 

37 

4 

73ABO 

380 

53 

5 

65ACL 

354 

57 

6 

73AAO 

327 

62 

7 

76ZPO 

302 

67 

8 

73KBO 

277 

73 

9 

73CAO 

275 

74 

10 

76ZEO 

246 

£3 

11 

i:  3RO 

239 

85 

12 

76Y50 

232 

88 

13 

76TDO 

203 

100 

14 

73BDO 

188 

108 

15 

52  BAA 

181 

112 

16 

52AOA 

179 

114 

17 

76Y20 

153 

133 

18 

73KKO 

135 

150 

19 

51ABC- 

133 

153 

20 

61CAO 

129 

157 

21 

73BKO 

115 

177 

22 

76WEO 

111 

183 

23 

51CCO 

106 

192 

24 

52BCC 

104 

195 

25 

76TCO 

98 

207 

26 

64BCE 

95 

214 

27 

64BCE 

89 

228 

28 

73DFO 

87 

234 

29 

63CAO 

84 

242 

Rank 

WUC 

Removals** 

MFHBR* 

30 

73CAP 

80 

254 

31 

76Y10 

79 

257 

32 

76WDO 

76 

267 

33 

51ABA 

64 

317 

*  Flight  Hours  -  20,317 
**  Type  1  +  Type  2  +  Type  6 
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This  document  establishes  Environmental  Stress  Screening  (ESS)  guidelines  and 
application/decision  criteria  for  equipment  already  operating  in  a  service  environ¬ 
ment.  The  objective  is  to  enable  the  Air  Force  Logistics  Center's  (ALC)  Item  Man¬ 
agers  to  determine  the  most  cost  effective  approach  to  introducing  ESS  on  field- 
repaired  equipment  and  to  establish  policy  for  inventoried  equipment  which  was  not 
subjected  to  ESS.  The  major  emphasis  of  this  guideline  document  is  to  establish  how 
and  where  ESS  can  be  most  practically  and  effectively  applied  within  the  current  Air 
Force  maintenance  organization  utilizing  the  AFM  66-1  documentation.  The  most 
significant  benefit  of  this  effort  to  the  Air  Force  is  the  potential  for  improving 
equipment  field  reliability  and  thereby  the  readiness  performance  of  the  end- item 
weapon  system,  as  well  as  significant  reductions  in  logistic  support  costs. 

Before  using  the  subject  guidelines,  it  is  extremely  important  to  appreciate  the 
differences  between  ESS  o'  new  and  in-service  or  inventoried  equipment.  On  new 
equipment,  the  designer  has  the  opportunity  to  include  in  the  equipment's  design  the 
test  criteria  for  all  of  the  environments  to  which  the  equipment  will  later  be 
subjected.  However,  inventoried  equipment  becomes  the  responsibility  of  the  Item 
Manager.  He  must  gather  the  informal’ ~n  on  the  equipment’s  environmental  qual¬ 
ification  capabilities,  its  past  and  current  operational  performance,  the  use  en¬ 
vironment,  and  its  reliaTJllity  characters  „cs.  The  latter  information  is  necessary  for 
the  Item  Manager  to  form  the  minimum  baseline  for  establishing  the  need  to  perform 
any  field  ESS  at  the  cognizant  ALC 

Field  ESS  is  an  end-item  Quality  Assurance  test  being  performed  in  a  non- 
homogeneous  product  control  environment.  Service  life  build-up  and  repeated  repair 
of  this  equipment  will  have  affected  lot  homogenity,  component,  and  lower  level  of 
assembly  process  controls,  as  well  as  identifiable  configuration  control.  The  objec¬ 
tive  of  this  guideline,  therefore,  is  to  establish  a  screening  program  which  will 
identify,  select,  and  optimally  screen  only  those  units  that  will  provide  the  most 
potential  for  reducing  the  aggregate  removal  rates.  These  rates  may  resu’t  from 


persistent  workmanship  defects  induced  either  in  the  initial  manufacturer's  process 
or  through  repeated  removal  and  repair  of  the  unit  during  its  operational  life. 


The  guideline  is  structured  to  provide  the  Item  Manager  with  a  step  by  step 
procedure  to  insure  that  only  the  most  technically  sound  and  cost  effective 
approaches  are  given  due  consideration.  As  shown  in  Fig.  C-l,  the  established  ESS 
selection  program  is  divided  into  four  major  areas: 

(1)  Equipment  Selection  -  based  on  existing  AFD056  and  MODAS  databases 

and  supporting  ALC  data,  establishes  equipment  population  and 
maintenance  rate  histories  sensitive  to  ESS 
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Figure  C-1.  Field  ESS  imp<wn«nt«tio.i  program  tatk  flow. 


(2)  Selection  Criteria  -  establishes  selection  means  to  minimize  the  quantity 
and  quality  of  the  equipment  selected  for  testing 

(3)  Test  Profile  Development  -  obtains  equipment's  environmental  qualification 
and  operational  capabilities,  and  establishes  temperature  and  vibration 
levels  and  durations  which  include  test  tailoring  practices  and 
considerations 

(4)  Economic  Selection  -  conducts  cost  studies  to  optimize  facility,  test  equip¬ 
ment,  manhours,  and  test  requirements  to  accurately  assess  all  costs  to 
determine  if  a  Return  on  Investment  (ROI)  savings  is  obtainable. 

The  task  flow  provided  in  Fig.  C-l  delineates  what  must  be  accomplished,  as  a  mini¬ 
mum,  to  develop  an  effective  and  economically  viable  program  as  discussed  in  the 
subsequent  paragraphs  of  this  guideline. 
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2.1  GOVERNMENT  DOCUMENTS 
Specifications 
MIL-M-38769 

Standards 
MIL- STD- 109 
MIL-STD-721 

MIL-STD-2164(EC) 


Handbooks 
MIL-HDBK-217 
MIL-HDBK-344  (USAF) 


Work  Unit  Code  Construction  and  Application 


Quality  Assurance  Terms  and  Definitions 
Definition  of  Terms  for  Reliability  and 
Maintainability 

Environmental  Stress  Screening  Process  for 
Electronic  Equipment 


Reliability  Prediction  of  Electronic  Equipment 
Environmental  Stress  Screening  of  Electronic 
Equipment 
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Analysis  of  Aeronautical  Equipment  Environmental 
Failures 
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Navy  Manufacturing  Screening  Program 


2.2  NON -GOVERNMENT  DOCUMENTS 

Institute  of  Environmental  Sciences  (IES) 

Environmental  Stress  Screening  Guidelines  for  Assemblies,  September  1984. 
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3  -  DEFINITIONS  &  ACRONYMS 
3.1  DEFINITIONS  APPLICABLE  TO  GUIDELINES 

AFD056  -  Output  process  data  tapes  and  reports  compiled  in  accordance  with  D056 
Data  System  directive  for  reporting  AF66-1  Maintenance  Data  Collection. 

Bad  Actor  or  Loser  Box  -  Units  of  product  which  have  a  frequency. of  removals 

greater  than  the  norm  or  process  average. 

j 

: 

i 

Consumable  -  An  item  of  material  used  up  beyond  recovery  in  the  use  for  which  it 
was  designed  or  intended. 

Defect  -  Any  non-conformance  of  the  unit  of  product  with  specified  requirements. 

Defective  -  Is  a  unit  of  product  which  contains  one  or  more  defects. 

Discrimination  Ratio  (DR)  -  The  measure  of  steepness  between  specified  MTBF(s)  and 
MTBR  to  discriminate  between  high  and  low  product  defective  potential.  Numerically 

•  — 1 S  1 ;  low  potential 
MTBR 

•  — B-^S)  >  1 ;  high  potential 
MTBR 


Environmental  Stress  Screening  (ESS)  -  A  series  of  environmental  tests  conducted  to 
disclose  weak  parts,  workmanship  defects  and  manufacturing  process  anomalies. 

ESS  Test  Profile  -  The  sequence  and  duration  of  stress  environments  under  which 
item  is  to  be  subjected. 

Failure  Free  Time  -  A  contiguous  period  of  time  (in  terms  of  environmental  cycles) 
during  which  an  item  is  to  operate  without  the  occurrence  of  a  failure  while  under 
environmental  stress. 


C-ll 


>•  • 


Failure  Rate  -  Reciprocal  of  MTBF. 


i 

•  i 

Field  ESS  -  ESS  performed  by  a  cognizant  equipment  ALC,  in  lieu  of  a  manufacture 
at  the  manufacturer's  facility. 

Inherent  Defect  -  A  failure  or  defect  that  is  a  function  of  the  intended  design  appli¬ 
cation  of  the  item,  when  operated  in  its  intended  operational  and  logistic  support  en- 
v’Vonment. 

Latent  Defect  -  A  process  induced  (manufacturing,  quality,  maintenance)  weakness, 
not  detected  by  ordinary  means,  which  will  either  be  precipitated  by  ESS  screening 
conditions  or  eventually  fail  in  its  intended  use  environment. 

Line  Replaceable  Unit  (LRU)  -  A  unit  normally  removed  and  replaced  as  a  single  item 
which  consists  of  assemblies  (SRUs),  accessories,  and  components  that  collectively 
perform  a  specific  functional  operation. 

Maintenance  Action  -  An  element  of  a  maintenance  event.  One  or  more  tasks  (e.g., 
removal,  fault  detection,  fault  isolation,  repair  and  inspection)  necessary  to  retain 
an  item  in,  or  restore  to,  a  specific  condition. 

Maintenance  Event  -  One  or  more  maintenance  actions  required  to  effect  corrective 
maintenance  due  to  any  type  of  failure,  or  malfunction,  false  alarm.  Categorization 
of  maintenance  events  based  on  the  D056  Air  Force  definition  Is  as  follows: 

•  Type  1  -  this  code  indicates  that  the  item  can  no  longer  meet  the  minimum 
specified  performance  requirements  due  to  its  own  internal  failure  pattern 

•  Type  2  -  this  code  indicates  that  the  item  can  no  longer  meet  the  specified 
performance  requirement  due  to  some  induced  condition  and  not  due  to  its 
own  internal  failure  pattern 

•  Type  6  -  this  code  indicates  that  maintenance  resources  were  expended  due 
to  policy,  modifications,  item's  location,  cannibalization,  or  other  'no  defect' 
conditions  existing  at  the  time  maintenance  was  accomplished. 

Mean  Time  Between  Failure  (MTBF)  -  The  mean  number  of  life  units  (i.e.,  operating 
hours,  flight  hours,  etc)  during  which  all  parts  of  the  item  perform  within  specified 
limits,  during  a  particular  measurement  interval  under  stated  conditions. 


Mean  Time  Between  Removals  (MTBR)  -  Total  number  of  system  life  units  (i.e.,  op¬ 
erating  hours,  flight  hours,  etc)  divided  by  the  number  of  removals. 

MODAS  -  The  Maintenance  and  Operational  Data  Access  System  (MODAS)  is  an  inter¬ 
active  Database  Management  System  (DBMS)  containing  24  months  of  field  and  depot 
reported  operational  (inventory,  status,  and  utilization)  and  maintenance  (AFR  66-1) 
data.  The  system  is  being  developed  primarily  to  support  ALC  System  Program  Man¬ 
agers  (SPMs)  and  Inventory  Managers  (IMs).  The  database  contains  summarized 
files  for  obtaining  Reliability  and  Maintainability  information  and  detailed  data  files 
such  as  non-mission  capable  hours  per  flight  hour,  sorties,  or  landings  by  Command 
or  Base. 

Percent  Defective  (d)  -  The  number  of  defective  units,  divided  by  the  number  of 
units  of  product. 

Process  Average  (u)  -  The  average  number  of  defects  per  unit  per  specified  interval 
of  time  (expressed  in  terms  of  removals  in  this  guideline). 

Removal  Rate  -  Reciprocal  of  MTBR. 

Removals  -  The  number  of  items  removed  from  a  system  during  a  stated  period  of 
time  as  related  to  demand  for  logistic  support,  and  excluding  removals  performed  to 
facilitate  other  maintenance,  removals  for  product  improvement,  and  removals  for 
cannibalization . 

Repairable  -  An  item  that  can  be  restored  to  perform  all  its  required  functions  by 
corrective  maintenance. 

Service  Life  -  The  duration  of  time  an  item  experiences  in  operational  inventory,  in¬ 
cluding  the  performance  of  any  maintenance  act  to  keep  the  item  in  operating  condi¬ 
tion. 

Shop  Replaceable  Unit  (SRU)  -  An  assembly  or  any  combination  of  parts,  subassem¬ 
blies,  and  assemblies  mounted  together,  normally  capable  of  independent  operation  in 
a  variety  of  situations  and  repairable  at  an  ALC. 


Specified  MTBF(s)  -  Design  or  operational  objective,  as  defined  by  handbook  pre¬ 
dicting  techniques  (i.e.,  MIL-HDBK-217),  specified  contractual  goals  (i.e.,  war¬ 
ranties),  field  operational  objectives  (i.e.,  R&M  2000  targets),  or  logistic  planners, 
goals  (i.e.,  wartime  loading  levels). 

System  -  A  group  of  interconnected  electronic  units  which  provide  a  specific  function 
(e.g.,  radar  system,  navigation  system,  etc). 

Tailoring  -  A  process  of  environmental  urveys  required  when  the  generic  ESS 
environmental  levels  exceed  the  unit's  functional  design  qualification  levels.  The 
potential  exceedance  levels  are  reduced  or  notched  at  resonant  frequencies  to 
eliminate  structural  and  intermittent  electrical  problems. 

Work  Unit  Code  (WUC)  -  An  alphanumeric  code  assigned  to  individual  systems,  sub¬ 
systems,  and  equipment  within  a  weapon  system  (aircraft,  ground  system,  missile, 
etc)  to  track  maintenance  activities. 

3.2  ACRONYMS  &  ABBREVIATIONS 
AFC  -  Amortized  Facility  Cost 
ALC  -  Air  Force  Logistics  Center 
ATE  -  Automatic  Test  Equipment 
DR  -  Discrimination  Ratio 
DTC  -  Direct  Test  Cost 
ESS  -  Environmental  Stress  Screening 
ICS  -  Integrated  Circuits 
LRU  -  Line  Replaceable  Unit 
LSC  -  Logistic  Support  Cost 
LSCS  -  Logistic  Support  Cost  Savings 
MTBF  -  Mean  Time  Between  Failure 
MTBR  -  Mean  Time  Between  Removal 
ROI  -  Return  on  Investment 
SRD  -  Standard  Reporting  Designation 
SRU  -  Shop  Replaceable  Unit 
WUC  -  Work  Unit  Code 


4  -  GUIDELINES 


In  a  field  ESS  scenario,  the  Line  Replaceable  Unit  (LRU)  provides  the  only 
basis  of  continuity  for  controlling'  the  non-homogeneous  state  of  field  hardware  con¬ 
figuration,  age,  and  maintenance  history.  The  justification  for  this  includes: 

•  The  LRU  is  the  focal  point  of  all  field  maintenance  reporting  and  provides 
reasonable  traceability  to  establish  quality  characteristics,  including  measur¬ 
able  removal  rates  and  individual  unit  traceability 

•  It  is  the  functional  basis  of  performance  for  which  support  equipment  is 
available  to  permit  complete  operational  diagnosis  of  any  performance  parame¬ 
ters 

•  The  integrated  functional  capability  permits  diagnosis  of  intermittent  work¬ 
manship  defects,  which  constitute  approximately  50°o  of  all  defects  encoun¬ 
tered  during  ESS  screening 

•  There  is  no  process  control  in  a  field  maintenance  scenario  that  can  replace 
all  levels  of  assembly  at  one  time  (short  of  an  overhaul),  and  provide  a 
homogeneous  quality  level.  On  a  per  repair  basis,  the  resulting  effects  of 
lower  level  assembly  ESS  become  inconsequential  and  are  highly  sensitive  to: 

-  The  true  number  of  defects  that  exist  in  the  LRU,  and  how  they  are 
distributed  in  the  lower  levels  of  assembly 

-  The  number  of  times  the  LRU  is  actually  repaired  in  its  lifetime 

•  Since  the  LRU  is  a  potential  collection  point  for  quality  related  problems, 
and  the  maintenance  reporting  is  identifiable  at  that  level,  it  provides  a 
means  for  monitoring  and  selecting  LRUs  that  tend  to  demonstrate  higher 
than  normal  removal  frequencies.  This  selection  process  provides  the  means 
for  managing  the  amount  of  testing  that  would  normally  be  required  if  all  the 
units  in  the  population  had  to  be  tested 

•  The  focal  point  for  Logistic  Support  Cost  Analysis  and  trade-off  is  the  LRU. 
If  potential  yields  are  not  sufficient  to  affect  the  removal  rates  improvement 
at  that  level,  the  effective  cost  benefit  Return  on  Investment  (ROI)  for  the 
ESS  testing  cannot  be  justified. 


fur  these  reasons  the  hardware  basis  of  these  guidelines  is  the  LRU,  and  the 
quantitative  measures  will  be  a  function  of  their  field  maintenance  removal  rate. 
Subsequent  selection  for  testing  at  lower  levels  of  assembly  (e.g.,  SRUs,  compo¬ 
nents)  will  be  a  function  of  the  LRU’s  selection  performance.  The  primary  processes 
of  selection  are: 

(1)  Establishment  Of  High  Contributor  Lists  (4.1) 

•  High  removal  rate  contributors 

(2)  Selection  Criteria  (4.2) 

•  Qualitative 

•  Quantitative 

•  High  bad  actor  selection 

(3)  Test  Profile  Development  (4.3) 

•  Environmental  characteristics 

•  ESS  profile 

•  LRU  ESS  requirements 
a  SRU  ESS  requirements 

•  ESS  tailoring 

•  Generic  ESS  test  recommendations 

(4)  Economic  Selection  Criteria  (4.4) 

•  Logistics  Support  Cost  Savings 

•  Test  Implementation  Cost 

•  Comparative  Return  on  Investment. 

The  objective  of  the  test  planning  will  be  achieved  when  the  highest  possible 
percentage  improvement  in  the  field  removal  rate  is  attained  by: 

•  Selecting  and  testing,  optimally,  the  lowest  number,  type,  and  kind  of  units 
possible 

•  Establishing  the  optimal  ESS  profile  and  duration  that  can  be  applied  to  a 
field  aged  unit,  without  compromising  equipment  life  or  test  stimulation  effec¬ 
tiveness  . 

A  high  level  of  success  is  thereby  insured,  within  the  lowest  test  risk  and  with 
minimized  test  implementation  cost,  while  improving  the  field  reliability  and  logistic 
support  cost  savings. 


4.1  ESTABLISHING  HIGH  CONTRIBUTOR  LISTS 

Prioritization  of  ESS  candidates  from  field  inventory  for  incorporation  into  the 
ALC  depot  repair  process  will  be  based  on  equipment  field  maintenance  histories  as 
compiled  from  the  Air  Force’s  D056  Field  Maintenance  Data  Base,  and  supplemental 
repair  data  as  maintained  by  the  equipment  responsible  ALC.  The  selection  criteria 
consists  of  those  indicators  and  parameters  which  can  be  used  for  tracking  equipment 
maintenance  performance,  and  enable  selection  of  equipment  or  groups  of  equipment 
and  lower  levels  of  assembly  that  offer  the  most  potential  for  producing  positive 
aggregate  improvement  to  both  the  equipment  and  the  weapon  system.  Figure  C-2 
provides  the  field  data  reduction  flow  necessary  to  provide  the  removal  accounta¬ 
bility  . 
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Figurs  C-2.  Field  data  reduction. 


4.1.1  High  Removal  Rate  Contributors 

High  removals  for  cause  provides  the  initial  basis  for  selection  of  candidate 
hardware . 

The  equipment's  high  removal  performance  is  based  on  those  Job  Control  Num¬ 
bered  (JCN)  actions  resulting  in  an  unscheduled  removal  (action  taken  codes 
(P,  R)),  resulting  in  the  following  maintenance  actions: 

•  Type  1  Maintenance  Actions  (as  defined  by  AFLCR66-15)  resulting  in: 

-  Repair  action  with  parts,  as  concluded  by  an  H  and  P  record 

-  Repair  action  without  parts 

-  Repair  action  resulting  in  NRTS  (Not  Repairable  This  Station) 

•  Type  2  (Induced  Maintenance)  Maintenance  Actions  (as  defined  by 

AFLCR66-15) 

•  Type  6  (Cannot  Duplicate)  Maintenance  Actions  (as  defined  by  AFLCR66-15), 

excluding  (How  Mai  Code  800  Series  Codes) 

-  Removals  for  access  and  upgrading 

-  Removals  for  cannibalization. 

Total  maintenance  removals  (R),  are  the  sum  of  the  type  maintenance  actions: 

(R)  =  Type  1  +  Type  2  +  Type  6 

4. 1.1.1  System  Selection  -  System  level  candidates  (e.g.,  radar  system,  navigation 
system,  etc)  will  be  drawn  from  those  categories  that  define  electronic  and  elec¬ 
tromechanical  systems,  equipment,  and  assemblies.  Typically,  Work  Unit  Code  (WUC) 
Categories  5  (Instrumentation/Navigation) ,  6  (Communications/Navigation),  and  7 
(Weapon  System)  for  aircraft  are  as  defined  per  specific  weapon  system  WUC 
manuals.  The  basis  of  total  removals  considered  will  be  summed  from  those 
categories. 

For  non-aircraft  systems  (ground  systems,  test  equipment,  missile  systems, 
etc),  the  WUC  designation  and  structure  are  as  defined  for  the  specific  Standard 
Reporting  Designation  (SRD)  code  for  that  system.  The  equipment,  LRU,  and  lower 
level  structure  are  as  defined  in  MIL-M-38769  (Work  Unit  Code  Construction  and 
Application) .  The  SRD  provides  the  access  code  for  selection  of  the  maintenance 
data  available  in  the  D056  and  MODAS  databases.  The  SRDs  for  all  reportable 
systems  are  provided  in  the  Maintenance  Data  Collection  System  Manual  TO-00-20-2. 
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The  WUC  structures  are  similar  to  those  of  aircraft  in  that  they  report  up  to  five 
digit  levels.  As  an  example,  the  AN/GKC-1(V)  Satellite  Tracking  Set  has  an  SRD  of 
(L4HG).  A  portion  of  the  WUC  structure  identified  for  this  system  as  reported  in 
MODAS  includes  the  alphanumeric  codes  defined  as  follows: 

AQXXX  (2  digit)  (System) 

AQEXX  (3  digit)  (Equipment) 

AQEPX  (4  digit)  (LRU  Level) 

AQEPH  (5  digit)  (SRU  Level) 

This  is  similar  to  aircraft  reporting,  with  the  difference  that  all  aircraft 
systems  start  with  numeric  prefixes. 

In  the  AFD056  data  system,  the  system  level  is  generally  defined  by  the  three 
digit  WUC  level,  or  to  the  level  which  will  appropriately  define  the  military  desig¬ 
nation  nomenclature  (e.g.,  AN/APQ,  ALQ,  etc).  The  prioritization  of  the  system 
will  be  by  its  ranking  based  on  total  removals.  Based  on  studies  of  high  density 
avionic  weapon  systems,  Fig.  C-3  provides  the  cumulative  average  removal  distribution 
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Figure  C-3.  Ayereqe  top  down  rank  dirtribution  of  electronic/avionic  lyrtem  &  LRUs. 
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for  equipment  and  LRU  rankings  by  removals.  Nominally,  90")  of  electronic  equipment 
removals  are  within  the  top  ten  ranked  by  removal.  This  is  true  of  the  non-aircraft 
systems  as  well,  and  is  a  function  of  tin-  type  of  equipment  being  described.  In  all 
eases,  only  -dee ironic  type  hardware  should  be  selected  far  ranking. 

!.'  !  '  ho'  fv-p'ace  ibl  ■  Uni'  ;  LRU)  Sd-vtion  -  <'  '.ndide*  LRUs,  typically  defined 
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!ti: gligiM*' .  This  will  provide  a  selection  within  the  top  do  LRUs  as’- noted  in  Fig. 
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'ion  "I'l:e  AFTO.n.  database  provides  lit  - 
traceability  of  SRUs.  They  are  normally  reported  at  the  five  digit  WUC 
are  generally  Vet  Repaire.ble  This  Station  (NRTS)  and  sent  to  a'  Cognisant- 


ALC  fra  repair  and  disposition.  Further-,  they  are  not,  or  are  rarely,  identifiable  ' 
by  s'  ■  .1  number  and  accountability  is  extremely  difficult.-  However,  SRUs -are  the 
eurivia  basis  for  ALC  repair  operations.  Thus,  selecting  an  SRU  for  ESS  should  be 
based  on  its  contribution  to  the  LRU's  removal  rate.  The  higher  the  frequency,  the 
more  significant  the  SRU.  -• 


i  h  are  C-4  j>rovides  a  decision  making  aid  for  identifying  candidate  SRU  or 
hnver  levels  of  assembly  effects  on  the  LRU  based  on  the  SRU's  removal  contribution. 
The  u'vissa  (X  as  is)  is  the  ratio  of  total  removals  contributed  by  the  SRU  or  lower 
level  of  assembly.  The  ordinate  (Y  axis)  on  the  right  side  of  the  graph  provides 
the  potential  improvement  achievable  by  the  SRU  or  lower  level  of  assembly  as  a 
result  of  screening.  A  100'.'  improvement  is  a  virtual  elimination  of  the  component  as 
a  removal  rate  contributor.  The  left  ordinate  is  the  corresponding  improvement  that 
is  achievable  at  the  LRU  level  based  on  the  achieved  SRU  or  lower  level  improvement. 
As  an  example,  if  an  SRU  or  lower  level  of  assembly  contributes  50%  of  the  total  LRU 
removals,  eliminating  it  completely  'or  100%  improvement)  cannot  improve  the  LRU 
any  more  than  50°o  since  you  cannot  gain  more  than  you  put  in.  Similarly,  if  the 
screen  of  the  SRU  is  only  50%  effective,  then  the  LRU  improvement  can  only  be  25%. 
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FiguraC-4.  Improvement  in  LRU  removal  rata  a  »  function  of  SRU 
removal  rata  contribution. 

For  these  reasons,  unless  it  can  be  ,'ustified  (based  on  the  graph  of  Fig.  C*4) 
that  the  rate  of  impact  at  a  lower  level  of  assembly  will  have  an  impact  at  the  LRU 
level,  there  is  no  gain  that  can  be  realized  by  screening  the  lower  level  of  assembly. 

A  candidate  SRU  should  be  selected  from  the  list  of  LRUs  developed  in 
Subsection  4. 1.1. 2.  A  rank  priority  list  of  these  SRUs  should  be  prepared  by  the 
cognizant  ALC.  The  percentage  of  repairs  performed  on  the  SRU  in  relation  to  all 
the  SRU  repairs  performed  against  the  LRU  should  provide  the  percentage  weight  for 
SRU  consideration. 

SRUj  RePairs _ x  100  =  po  SRU;  contribution 

Total  SRU  Repairs  per  LRU 


This  percentage  is  the  ratio  that  could  be  used  in  Fig.  r-4  in  lieu  of  the  removal 
rate  ratio.  The  frequency  of  repair  data  for  the  SRU  will  be  developed  from  the 
ALC  repair  data  bases. 


4. 1.1. 4  Lower  Levels  of  Assembly  Selection  -  Without  a  complete,  detailed  opera¬ 
tional  history  of  all  typical  parts  and  lower  leve.s  of  assembly,  there  is  no  basis  of 
selection  of  lower  levels  of  assembly  for  testing  in  the  field.  Testing  at  this  ievel 
should  be  performed  at  a  supplier  or  vendor  facility,  so  as  to  screen  (as  an  in- 
procer.  function)  logistically  supplied  material  that  will  enter  previously  screened 
higher  order  assemblies  (LRUs  or  SRUs).  Any  consideration  of  screening  existing 
stocks  of  lower  levels  of  assembly  should  be  closely  cost  evaluated.  Lower  levels  of 
assembly  include  hybrids,  Integrated  Circuits  (ICs)  and  assorted  piece  parts.  The 
basis  for  selection  and  screening  at  these  levels  may  be  in  accordance  with  DOD- 
HDBK-344  (USAF).  Once  again,  the  basis  for  traceability  shall  be  the  candidate 
LRUs. 

4.2  SELECTION  CRITERIA 

T'ae  selection  criteria  will  be  applied  to  the  candidate  population  of  high  removal 
LRUs  selected  in  Subsection  4. 1.1. 2,  to  provide  an  optimal  selection  basis  which  will 
further  minimize  and  eliminate  hardware  which  will  not  be  ESS  sensitive. 


4.2.1  Qualitative  Criteria 


Qualitative  criteria  shall  assess  the  physical  and  design  attributes  of  the  hard¬ 
ware,  and  will  require  the  application  of  engineering  judgement,  and  design  and  lo¬ 
gistic  support  knowledge  of  the  hardware  to  support  decisions  to  continue.  Quali¬ 
tative  factors  to  be  considered  include: 

•  Functional  Testability  -  effective  ESS  requires  functional  verification  during 
and  after  testing 

-  Special  Test  Equipment  (e.g.,  ATE)  and  loading  demands 

-  Use  of  smaller,  limited  range,  field  type  testers 

•  Previously  ESS  Te:  ted  LRUs  -  ESS  testing  affects  design  service  life;  multi¬ 
ple  testing  of  the  same  units  can  be  destructive 

•  Environmental  TestabiMty  -  state  of  the  art  in  facilities  and  equipment 
availability  may  ba  a  limiting  factor  in  handling  the  test  article;  need  to 
evaluate: 
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•  Instrumentation  requirements 

-  Weight  of  unit 

-  Size  of  unit 

-  Chamber  capacities  and  rates  of  change 

-  Vibration  table  and  surface  capacity 

•  Level  of  Repair  -  test  repairables  only;  non-repairables  should  be  tested  as 
procured,  preferably  at  the  vendor's  facility 

•  Fixture  and  Mounting  Design  -  fixture  and  mounting  designs  should  be  uni¬ 
versal.  Special  purpose  designs  for  specific  hardware  applications  could  be 
expensive 

•  Logistic  Availability  -  ESS  testing  will  stimulate  failures  at  higher  rates  than 
normal.  Any  special  repair  requirements  and  considerations  (e.g.,  out  of 
production  status)  could  be  expensive,  and  could  create  unplanned  demands 
(e.g.,  hardware  that  never  failed  before  may  start  failing) 

•  Service  Life  -  ESS  testing  will  affect  service  life  from  both  an  effective 
yield  and  cost  benefit  potential  point  of  view,  and  has  the  potential  of  being 
a  destructive  test.  Note;  successful  ESS  does  not  extend  service  life 

•  Configuration  Control  -  multiple  configurations,  component  interchangeability 
levels,  functional  variations,  lot  homogeneity  and  mixed  requirements  can 
create  wide  variations  in  testability  for  the  same  type  hardware 

•  Warranty  Commitments  -  ESS  at  an  ALC  may  affect  any  contractual  warran¬ 
ties;  considerations  may  be  to  perform  ESS  on  bad  actors  as  part  of  war¬ 
ranty 

•  Commercial  Grade  Hardware  -  commercial  grade  hardware  is  testable,  how¬ 


ever,  failure  and  warranty  responsibility  may  be  a  problem 
•  Vintage  Design  -  qual  levels  may  require  extensive  tailoring,  to  the  extent 
that  ESS  profile  is  ineffective  or  potentially  destructive. 

4.2.2  Quantitative  Criteria 

Candidate  LRUs  should  have  a  Discrimination  Ratio  (DR)  of  specified  MTBF(s) 
to  actual  removal  rate,  MTBR,  of  greater  than  one: 

DR  =  MTBF(,s)  =>1 
MTBR 


MTBF(s)  is  the  iesign  or  operational  objective,  as  defined  by  handbook  predicting 
techniques  (e.g.,  MIL-HDBK-217) ,  specified  contractual  goals  (e.g.,  warranties), 
field  operational  objective  (e.g.,  R&M  2000  targets),  or  logistic  planning  goals  (e.g., 
wartime  loading  levels)  and  should  be  expressed  or  factored  in  the  time  variables  of 
the  operating  system  (e.g.,  flight  hours,  operating  hours,  etc).  The  actual  mea¬ 
sured  rate,  expressed  as  the  MTBR,  is  the  system  or  equipment  operating  time 
parameter  divided  by  the  total  removals. 


This  relationship  will  have  more  significance  with  non- aircraft  type  systems  and 
low  density  (one  of  a  kind)  production  systems.  The  measurements  of  MTBF  or 
MTBR  will  then  provide  the  probable  departure  from  the  specified  value,  and  are 
really  the  only  indication  that  workmanship  defects  may  exist.  The  operating  time 
parameters  should  be  either  the  recorded  operating  hours  of  the  unit  or  system,  or 
duty  cycle  weighted  in  service  hours.  As  an  example,  if  the  ownership  time  is  one 
year,  that  would  be  a  total  of  8760  hours  (365  days/year  x  24  hours/days).  If  the 
operational  duty  cycle  of  the  unit  or  system  is  eight  hours  per  day,  then  the 
effective  operating  time  would  be  2920  hours  (1/3  x  8760).  This,  divided  by  the 
reported  removals  or  failures  for  the  period,  provides  the  measured  MTBR  value  to 
be  compared  in  the  discrimination  ratio. 

4.2.3  Bad  Actor  Selection  Criteria 


In  establishing  a  bad  actors  program,  the  objective  is  to  minimize  the  number  of 
units  that  should  be  tested,  and  identify  only  those  units  which  will  provide  the 
most  potential  benefit  from  an  ESS  effectiveness  point  of  view.  These  units  will 
have  a  higher  frequency  of  removals  than  the  norm,  usually  in  the  form  of  false 
alarms  or  cannot  duplicate  conditions,  implying  the  potential  for  intermittent  con¬ 
ditions.  The  serialization  process  of  LRUs  as  reported  in  the  D056  database  provides 
the  means  for  identifying  and  determining  the  distribution  of  removals  on  a  per  unit 
basis.  The  process  for  establishing  bad  actors  is  shown  in  Fig.  C-5. 


4. 2. 3.1  Establishing  a  Process  Average  (y)  -  The  process  average  (u),  or  the 
average  number  of  defects  per  unit  per  period  of  time  is  expressed  as  the  ratio: 
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Figure  C-5.  Bad  xrtof  relection. 


where:  R  =  the  total  removals 

n  =  number  of  weapon  systems  reporting 
k  =  number  of  like  units  per  weapon  system 
and  (n.k)  £  N,  which  represents  the  minimum  number  of  units  that  should  be  re¬ 
porting  (whether  they  have  removals  or  not). 

The  value  (n.k)  represents  the  least  number  of  units  that  should  be  reporting 
in  the  period  (T)  in  contrast  to  the  actual  number  of  units  (N).  In  Fig.  C-5,  the 
number  of  weapon  systems  reporting  is  n  =  236,  and  the  number  of  units  per  system 
is  k  =  1;  therefore  n.k  =  (236)  (1)  =  236. 

•  The  actual  number  of  units  reporting  (N)  =  236 

•  The  total  removals  (R)  for  the  period  (1  year)  =  345 

•  The  process  average  p  =  345  =  1.5 

236 

From  the  frequency  distribution.  Fig.  C-5  (B),  it  is  seen  that  80%  of  all  units  are 
1.5  defects  per  unit  or  less.  This  means  that  189  units  (0.80  x  236)  are  at  the 
process  average  or  less.  The  remaining  units  (236-189=47)  will  be  greater  than  the 
process  average. 

4. 2. 3. 2  Rank  Serialized  LRUs  -  Ranking  the  serial  numbers  reporting  by  the  num¬ 
ber  of  removals  reported  per  serial  number  (top  down)  identifies  the  units  falling 
above  and  below  the  process  average.  Essentially,  as  shown  in  Figure  C-5  all  units 
with  two  or  more  removals  (D  =  2)  are  in  excess  of  the  p**ocess  average  (p  =  1.5). 
This  identifies  serial  number  68  and  above  as  the  high  contributors. 

4. 2. 3. 3  Establish  Defect  Criteria  -  The  defect  criteria  (D)  is  the  real  removal  value 
or  greater  that  exceeds  the  process  average  (p)  to  the  next  highest  whole  value: 

D  >  p  =  unit  bad  actors  frequency 
In  Fig.  C-5,  the  value  is: 

D >  p  =  D>  1.5  =  2.0 


The  number  of  units  with  two  defects  per  unit  per  year  or  more  is  68;  or  68/236  = 
29°o.  This  group  would  represent  the  most  likely  population  of  LRUs  with  the  high¬ 
est  potential  for  ESS  effectiveness. 


4. 2. 3. 4  Predicting  Outcome  -  The  process  average  (y)  and  the  number  of  LRUs 
that  have  (D)  or  greater  defects  can  be  used  to  anticipate  or  project  a  potential  im¬ 
provement  rate  assuming  that  the  defective  process  average  (v^)  will  effectively 
achieve  or  approach  the  good  process  average  (y^)  as  a  result  of  effective  ESS. 
The  percent  gain  would  then  De: 

v  ~  ur 

%  gain  =  p  x  100 

When  tested,  defective  units  of  process  average  (y^)  will  improve  to  achieve  or  ap¬ 
proach  the  good  process  average  y^,  and  the  overall  process  average  will  at  least 
approach  y  =  >  y^. 

To  demonstrate  this,  consider  again  the  illustration  in  Fig.  C-5.  The  data  from 


the  rankings 

is  tabulated  below: 

No.  Units 

No.  Removals 

Process  Average  y 

Defective  (D) 

68 

177 

2.6 

Good  (G) 

168 

168 

1.0 

236 

345 

1.5 

The  defective  process  average  y^  =  2.6,  the  good  process  average  yQ  =  1.0, 
and  by  testing  the  68  units  it  is  expected  that  yjj-fr-y^j-^-1.0.  This  will  result  in 
an  overall  process  average  y-*-yQ-+-1.0.  The  percent  gain  potential  is  therefore: 

%  Gain  =  -•5--  —  =  33% 

1.5 

or  the  rate  improvement  factor  is  1.3  (the  operational  MTBR  could  improve  by  a  1.3 
factor) . 


4.2.4  LRU  Age 

Experience  with  LRU  age,  in  terms  of  years  of  service  and  the  potential 
effects  of  ESS,  does  not  exist  to  any  formal  degree  or  with  sufficient  background  to 
support  decisions  one  way  or  the  other.  Factors  affecting  LRU  age  include  aspects 
of  growth  as  a  result  of  years  of  reliability  improvement  and  upgrading,  as  well  as 
degradation  as  a  result  of  extended  use  and  potential  wearout.  These  factors  must 
be  counterbalanced  to  rationalize  the  potential  effect  from  ESS  which  in  one  case  can 
significantly  improve  the  reliability  of  the  device,  and  another  case,  degrade  it  in 
that  it  can  potentially  be  destructive  to  the  equipment.  Although  reliability  may 
improve,  ESS  does  not  improve  service  life. 

Grwoth  extrapolations  developed  from  non-ESS  and  ESS  population  performance, 
which  together  span  some  15  years  of  compiled  data,  have  concluded  a  generic  curve 
as  depicted  in  Fig.  C-6.  It  is  derived  from  a  weighted  consideration  of  all  the 
growth  curves  developed,  both  positive  as  well  as  negative,  and  describes  the 
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potential  percent  reduction  in  the  total  rate  as  a  function  of  performing  ESS  on 
boxes  having  anywhere  between  0  and  25  years  of  field  life.  Based  on  Fig.  C-6, 
LRUs  with  an  operational  field  service  exceeding  10  years  offer  little  in  the  way  of 
improvement  benefit  due  to  ESS. 

However,  overhauled  units  which  are  refurbished  for  the  purpose  of  extending 
life  should  be  subjected  to  ESS  irrespective  of  age.  This  will  insure  that  reliability 
enhancements  of  refurbished  components  will  be  effective.  Tailoring  may  be  a  factor 
in  developing  proper  environmental  profiles  for  these  LRUs. 

4.3  ^EST  PROFILE  DEVELOPMENT 

At  this  point  in  the  ESS  candidate  selection  process,  the  Air  Force  Item  Man¬ 
ager  has  already  completed  the  field  data  analysis  (Section  2)  and  equipment  selec¬ 
tion  criteria  (Section  3).  The  next  step  is  to  determine  the  equipment  level,  i.e., 
LRU  or  SRU,  to  which  the  ESS  testing  will  be  applied,  and  also  the  specific 
environments  necessary  to  stimulate  the  potential  workmanship  and  manufacturing 
process  problems.  Figure  C-7  describes  the  process  to  define  the  specific  testing 
effort  that  will  be  estimated  during  the  Final  Cost  Analysis  described  in  Subsection 
4.4. 
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Figure  C-7.  nviroomtntal  twt  program  flow. 


During  electronic  equipment  development  programs,  the  equipment  designer  se¬ 
lects  numerous  environmental  tests  to  evaluate  the  equipment's  operational  perfor¬ 
mance  at  its  lower  levels  of  assembly  to  determine  as  early  as  possible  whether  there 
are  any  potential  design,  manufacturing,  process  and/or  any  workmanship  problems. 
It  is  crucial  at  that  time  to  ascertain  that  the  purchased  parts  and  the  subassembly 
designs,  when  assembled,  meet  the  design's  predicted  environmental  criteria.  This 
environmental  test  verification  phase  in  the  equipment's  development  is  very  cost 
effective  because  it  exposes  design  problems  when  there  is  still  the  opportunity  to 
resolve  them.  At  that  time,  the  current  experts  of  all  the  technical  communities  will 
be  familiar  with  the  design  and  easily  resolve  any  problem.  When  satisfied  with  the 
design,  all  further  acceptance  testing  is  at  the  highest  level  of  assembly,  the  LRU. 

Once  the  LRU  is  delivered  to  the  customer,  any  delta  ESS  to  verify  field  cor¬ 
rective  action  should  continue  at  that  level.  It  is  at  this  level  of  assembly  that  the 
required  support  equipment  is  already  designed  and  manufactured,  permitting  com¬ 
plete  operational  diagnosis  of  any  LRU  performance  parameters  and  thus  minimizing 
test  time  and  costs.  Fully  operational  LRU  ESS  installations  are  mandatory  because 
approximately  50°  of  all  workmanship  related  problems  are  of  an  intermittent  nature. 

4.3.1  Environmental  Characteristics 

The  next  step  in  the  ESS  process  prior  to  conducting  any  environmental  tests 
on  the  candidate  equipment  is  to  determine  the  environmental  characteristics.  Re¬ 
search  is  required  to  obtain  data  on  the  equipment's  environmental  qualification  tests 
results,  including  any  environmental  acceptance  tests  levels  and  the  operational  data 
describing  the  vehicle  environment  in  which  the  equipment  is  currently  installed. 

4.3.2  ESS  Profile 

Since  these  guidelines  are  directed  towards  Government  Furnished  Equipment 
(GFE) ,  non-ESS  field  electronics  which  are  normally  repaired  at  a  depot,  it  is  ad¬ 
vantageous  to  utiMze  a  generic  ESS  profile.  This  approach  simplifies  the  ESS  test 
operations,  reduces  equipment  costs,  and  minimizes  the  ESS  familiarization  and  train¬ 
ing  requirements.  Thus  we  recommend  using  the  ESS  requirements  and  procedures 


defined  in  MIL- STD-2164  (EC).  This  document  is  based  on  generic  test  levels  and 
durations  developed  for  space,  aircraft,  and  shipboard  electronics.  The  consistent 
utilization  of  these  requirements  has  proven  that,  in  most  electronic  equipment  de¬ 
signs,  there  is  a  minimum  structural  rigidity  existing  within  the  pre-1980  designs 
which  represents  the  major  number  of  electronic  equipment  in  the  field  today.  It 
has  been  shown  that  equipment,  though  never  exposed  to  random  vibration,  has 
demonstrated  the  required  structural  and  electronic  integrity  to  be  inherent  within 
the  design.  Thus  the  only  problems  that  should  appear  during  ESS  testing  are 
those  directly  associated  with  workmanship  and/or  manufacturing  process. 

4.3.3  LRU  ESS  Requirements 

The  total  ESS  program  includes  physical  inspection,  functional  tests,  and 
periods  of  environmental  exposure  designed  to  stimulate  latent  defects  without  incur¬ 
ring  equipment  fatigue  damage.  Figure  C-8  presents  the  overall  test  flow  that  will 
be  used  to  verify  that  an  equipment  is  ready  for  operational  use. 
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4. 3. 3.1  Individual  Tests  -  Each  equipment  under  test  will  be  subjected  to: 

•  Initial  Operational  Test  -  an  equipment  operational  test  in  accordance  with 
the  seller- prepared  test  procedure  will  be  performed,  and  data  will  be  re¬ 
corded  to  verify  that  the  equipment  fully  complies  with  detailed  performance 
requirements.  The  test  procedure  will  include  measurements  required  for  a 
quantitative  assessment  of  all  functional  performance  parameters.  GO/NO  GO 
evaluation  is  acceptable.  The  record  for  pretest  data  will  be  retained  for 
use  as  a  reference  during  subsequent  ESS  tests 

•  Environmental  Test  -  equipment  submitted  for  test  will  be  subjected  to  a 
fixed  duration  ESS  test.  The  operational  equipment  will  be  continuously 
monitored,  and  all  functional  parameters  will  be  exercised  repeatedly  at  the 
highest  attainable  rate.  The  mechanization  of  the  functional  check-out  and 
its  speed  of  repeatability  will  represent  a  major  task  in  the  overall  formu¬ 
lation  of  the  ESS  test  program.  All  vibration  testing  will  be  conducted  with 
the  equipment  h^rd- mounted,  regardless  of  whether  or  not  it  is  to  be  in¬ 
stalled  on  vibration  isolators  in  its  use  environment. 

Each  equipment  will  be  exposed  to  random  vibration  and  thermal  cycling 
periods  (Fig.  C-8).  Since  the  purpose  of  this  test  is  to  eliminate  latent 
manufacturing  defects,  all  defects  detected  during  this  test  will  be  recorded 
and  repaired 

Note:  Since  this  testing  is  directed  to  in-service  GFE,  exposed  to  numerous 
environmentally  operational  hours,  the  need  for  a  fixed  ESS  failure -free  per¬ 
iod  is  not  necessary.  Thus  as  long  as  each  ESS  related  problem  is  corrected 
when  it  occurs,  it  is  expected  that  at  the  completion  of  the  fixed  during  ESS 
periods  there  will  be  a  justifiable  amount  of  failure-free  operation 

o  Vibration  -  the  equipment  in  an  operating  mode  (power  on)  will  be  exposed 
to  one  five  minute  burst  of  random  vibration  in  the  axis  deemed  most  sus¬ 
ceptible  to  vibratory  excitation.  Failures  occurring  during  this  five  minute 
test  will  be  corrected  as  they  occur.  The  random  vibration  spectrum  will 
be: 

20-80  Hz  at  3  dB/octave  rise 
80-350  Hz  at  0.04gVHz 
350-2000  Hz  at  3  dB/octave  rolloff 

•  Thermal  Cycling  -  the  equipment  in  an  operating  mode  (power  on)  will  be 
subjected  to  a  thermal  cycling  test  for  a  period  of  40  hours  in  accordance 
with  the  appropriate  cycle  (Fig.  C-9).  The  required  number  of  thermal 
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Figure  C-9.  Temperature  cycling  profile  for  ambient  cooled  &  lupplementally  cooled  equipment. 


cycles  should  be  interrupted  for  each  repair  action.  The  thermal  limits 
(high  and  low  temperature  extremes  of  chamber  air)  for  cycling  will  be  those 
values  of  operational  temperature  requirements  defined  by  the  equipment 
specification . 

4. 3. 3. 2  Final  Functional  Operational  Test  -  Upon  successful  completion  of  the  en¬ 
vironmental  testing,  a  final  functional  test  will  be  performed  at  room  ambient  con¬ 
ditions  to  verify  satisfactory  operation  of  the  equipment  in  accordance  with  the 
parameters  specified  in  the  prime  item  specification.  Operational  measurements  will 
be  compared  with  those  obtained  during  the  initial  operational  test  and  evaluated 
based  upon  the  specified  acceptable  functional  limits. 

4.3.4  SRU  ESS  Requirements 

ESS  testing  of  SRUs  should  be  limited  to  spares  and  those  items  which  have  ex¬ 
perienced  high  failure  rates  in  the  operational  environment.  For  those  situations,  no 
ESS  testing  should  be  imposed  across  the  board.  When  necessary,  the  optimum  en¬ 
vironmental  test  should  be  a  non-operating  thermal  shock  for  a  period  of  25  repeti¬ 
tive  cycles.  The  maximum  transfer  time  should  be  one  minute  or  less  and  the  soak 
periods  approximately  one  hour  each.  The  temperature  extremes  should  be  based  on 
the  maximum,  not  operating,  temperature  qualification  level.  Only  those  assemblies 
which  are  known  problem  installations  should  be  ESS  tested.  If  a  spares  manufac¬ 
turer  is  producing  a  good  product,  which  indicates  his  workmanship  is  good,  the 
manufacturer  should  not  be  burdened  with  additional  costly  requirements.  Test  ef¬ 
forts  should  concentrate  on  the  "bad"  products. 

4.3.5  ESS  Tailoring 

In  the  case  of  inventoried  equipment,  consideration  must  be  given  to  the  en¬ 
vironmental  capabilities  of  the  ESS  candidate  equipment.  The  probability  exists  that 
this  equipment  may  not  have  been  designed  to  function  in  a  generically  defined  ran¬ 
dom  vibration  or  rapid  thermal  cycling  environment.  Thus  some  form  of  environ¬ 
mental  tailoring  is  required.  It  should  be  pointed  out  that  experience  has  taught  us 
that  this  does  not  mean  the  subject  equipment  is  incapable  of  withstanding  the  latter 
environment.  In  most  cases  this  same  equipment  probably  has  been  operating  suc¬ 
cessfully  in  a  similar  environment,  i.e.,  in  a  jet  aircraft  experiencing  rapid  tem¬ 
perature  changes  and  random  vibration  without  any  structural  damage.  The  only 

potential  problem  for  some  equipment  was  that  when  the  generic  ESS  levels  were  ap¬ 
plied,  performance  anomalies  (such  as  out  of  tolerance  conditions)  became  evident. 


In  order  to  accommodate  this  situation,  ESS  levels  must  be  tailored  by  notching 
the  random  vibration  spectrum  at  the  primary  LRU  resonances.  For  this  vibration 
notching,  accelerometers  must  be  installed  at  selected  items  within  the  equipment  and 
a  series  of  low  level  sinusoidal  sweeps  performed  to  define  resonance. 

After  instrumenting  these  test  units,  it  became  apparent  that  if  the  response 
acceleration  exceeded  an  amplification  factor  of  10,  a  potential  performance  problem 
would  become  evident.  In  these  instances  it  was  necessary  to  tailor  the  random 
vibration  levels  by  notching  certain  frequencies  to  minimize  the  operational  problems 
associated  with  marginal  component  or  subassembly  installation. 

A  successful  technique  during  tailoring  is  to  install  adequate  instrumentation  on 
the  test  article  to  record  the  amplification  factors  measuring  a  potentially  sensitive 
component  installation,  during  a  ±3.0 g  sinusoidal  sweep  from  20  to  2000  Hz.  At 
those  resonances  where  the  amplification  exceeds  10,  random  spectrum  notching  is 
performed  to  reduce  the  input  to  the  equipment.  An  example  of  this  technique  is  as 
follows: 

(1)  Conduct  a  ±3g  sine  sweep  and  measure  the  ratio  (response/control)  at  the 
desired  locations  (see  Fig.  C-10  for  typical  response) 


VIBRATION  SURVEY  RUN  8Z-3G  SINE  SWEEP  - 
TRANSMISSI8ILITY  PLOT  OF  ACC  9  -  A3000  TRAY 
IN  Z  AXIS 
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Figure  C-IO.Typical  ±  3g  tine  sweep  response. 


Record  measured  vibration  amplification  factors  exceeding  10: 


Acc  No. 


Location 


1  Bottom  of  A1000  assembly  11  @  104  Hz 

3  A4000  Tray  16  @  104  Hz 

9  A3000  Tray  15  &  80  Hz 

10  A8000  Tray  20  &  100  Hz 

13  Top  of  AS000  assembly  16  @  103  Hz 

7  A5000  assembly  10  0  196  Hz 

Perform  random  vibration  notching  to  reduce  amplifications  to  10: 

79  to  95  Hz  -  Max  Ampl  =  15  PSD  =  (10/15)  x  .04  =  .0267  GSQD/Hz 

95  to  104  Hz  -  Max  Ampl  =  20  PSD  =  (10/20)  x  .04  =  .J200  GSQD/Hz 

104  to  113  Hz  -  Max  Ampl  =  14  PSD  =  (10/14)  x  .04  =  .0286  GSQD/Hz 

Above  113  Hz  No  notching  done  in  this  freq  range. 

Incorporate  notching  on  the  generic  6.0g  RMS  random  vibration  test  spec¬ 
trum  as  follows  in  Fig.  C-ll: 
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Figure  C-11.  Notching  on  6.0g  RMS  random  vibration  test  ipectrum. 


With  respect  to  tailoring  the  rapid  thermal  cycling  requirements,  the  tempera  - 
ure  extremes  were  selected  based  n  the  equipment's  operational  environmental 
ualification  limits.  It  was  also  determined  that,  at  the  LRU  level,  there  was  not 
ny  advantage  to  exceeding  the  5°C/minute  rate  of  change. 


What  we  did  see  in  our  field  equipment  ESS  investigations  indicated  there  was 
ittle  advantage  to  performing  both  a  40  hour  pre- defect  free  and  a  40  hour  defect 
ree  thermal  cycling  test.  The  test  data  indicated  that  the  problems  which  the 


previous  use  environment  stimulated  were  the  major  workmanship  and  manufacturing 
problems,  thus  negating  the  need  to  do  a  pre-defect  period.  It  was  thus  decided 
that  if  the  candidate  equipment  experienced  an  ESS  failure  during  thermal  cycling 
and  was  replaced  immediately  after  that  cycle,  there  should  be  an  adequate  number 
of  defect  free  cycles  accrued  within  the  fixed  40  hour  period.  This  considerably 
reduces  the  test  time  and  associated  costs. 


4.3.6  Generic  ESS  Test  Recommendations 

ESS  tests  of  contract  end  items  consisting  of  electronic  components  and  subas¬ 
semblies  should  be  subjected  to  the  rapid  thermal  cycling  and  random  vibration  de¬ 
fined  in  MIL-STD-2164(EC) .  The  objective  of  these  tests  is  to  stimulate  early  occur¬ 
rence  of  workmanship  and  manufacturing  process  problems  so  that  corrective  action 
can  be  addressed  prior  to  delivery  of  the  procuring  activity. 

During  the  specified  random  vibration  and  rapid  thermal  cycling  tests,  it  is 
mandatory  that  the  equipment  be  operated  and  monitored  to  the  fullest  extent  possi¬ 
ble,  except  during  the  down  portion  of  the  thermal  cycle.  It  should  be  noted  that 
the  equipment  will  experience  the  required  thermal  stresses  from  the  removal  of 
equipment  power  and  the  rapid  descent  of  the  temperature  chamber. 

Equipment,  Box,  or  Drawer 


Random  Vibration 
Power  spectral  density 


Axes  stimulated 

Duration  of  vibration 

Power  on/ (equipment  operation) 

Equipment  monitoring 

Thermal  Cycling 


_ (LRU/LRM) _ 

20-80  Hz  @  +3dB/Octave 

80-350  Hz  @  .04  g2/Hz 

350-2000  Hz  @  -3dB/Octave 

One  axis  perpendicular  to  the  printed 

wiring  board 

5  min  at  start  of  test 


•  Temperature  range 

•  Temperature  rate  of  change 

•  Temperature  dwell  duration 


Operating  environmental 
qualification  limits 
5°C/min 

Based  on  thermal  signature  study 


40  hours 

Yes 

Yes 


•  Thermal  cycling  duration 

•  Power  on/ (equipment  operating) 

•  Equipment  monitoring 

•  Electrical  testing  after  ESS  Yes  @  room  ambient 

4.4  ECONOMIC  SELECTION  CRITERIA 

The  cost  benefit  will  be  based  on  the  financial  return  when  money,  time,  and 
effort  is  invested  to  perform  the  ESS,  and  the  resulting  annualized  ROI  is  at  least 
33-.l/3°o.  The  equation  in  Fig.  C-12  defines  the  economic  elements  required  to  per¬ 
form  the  analysis  and  develop  projected  cost  savings  and  ROI.  Each  LRU  candidate 
selected  for  field  ESS  should  be  cost  benefit  evaluated  to  support  test  justification. 
The  task  methodology  for  this  process  is  shown  in  Fig.  C-13.  Although  the  meth¬ 
odology  for  determining  the  cost  is  applicable  to  any  level  of  assembly,  i.e.,  LRU, 
SRU,  and  subassembly,  it  is  recommended  that  cost  justification  be  rationalized  at 
the  LRU  level,  since  significant  net  cost  savings  and  ROI  become  more  difficult  to 
achieve  because  of  lower  unit  cost  levels  and  higher  element  MTBF  values.  If  the 
improvement  and  ROI  cannot  be  significantly  rationalized  at  an  LRU  level,  it  is 
probably  not  worth  considering. 

4.4.1  Logistics  Support  Cost  Savings  (LSCS) 

The  LSCS  is  expressed  as: 

LSCS  =  LSC  -  LSC 
o  p 

where : 

(1)  LSCq  -  Baseline  logistic  support  cost  is  the  current  operational  LSC  of 
the  LRU  at  the  time  of  selection 

(2)  LSCp  -  The  projected  LSC  that  will  be  realized  as  a  result  of  the  removal 
rate  improvement. 

The  LSC  analyses  should  be  performed  with  the  models  and  scenarios  developed 
for  the  specific  weapon  system  (e.g.,  fighter  aircraft,  transport,  stationary  ground, 
mobile  ground)  from  which  the  LRU  is  being  selected. 

4. 4. 1.1  LSCS  Estimating  -  For  estimating  purposes.  Fig.  C-14  is  provided.  The 
LSCS  factor  is  read  directly  as  a  function  of  the  current  operational  removal  rate 
per  month  for  the  specific  LRU,  as  obtained  from  the  maintenance  data,  and  the  pro¬ 
jected  rate  improvement  either  as  a  direct  input  from  Subsection  4. 1.1. 4  or  adjusted 


(LSCS)  -  ((AFC)  *  (DTC)) 


ROI%  - - X10C  >  33  1/3% 

((AFC)  ♦  (DTC1)  •  (SL  •  EA) 


WHERE: 


(1 )  LSCS  -  LOGISTICS  SUPPORT  COST  SAVINGS 

(2)  (AFC  +  OTC)  -  TEST  IMPLEMENTATION  COST 

•  AFC  -  AMORTIZED  FAULITY  COST  (NON-RECURRING) 

•  DTC  -  DIRECT  TEST  COST  (RECURRING) 

(3)  [(LSCS)  -  (AFC  *  DTC)!  -  NET  COST  SAVINGS 

(4)  (SL- EA!  •  REMAINING  SERVICE  LIFE 

•  SL  *  DESIGN  SERVICE  LIFE 

•  EA  -  EQUIPMENT  AGE  AT  THE  TIME  ESS  IS  PERFORMED' 
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Figure  0-12.  ESS  ROI  formulation. 
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Figure  C-14.  Projected  field  ESS  logijtic  support  cost  savings  factor  vs  MTBM  improvement  factor. 

as  a  function  of  the  average  equipment  age  at  the  time  of  selection  (equipment  age  is 
defined  as  the  average  age  of  the  population  of  candidal  LRUs) .  The  LSCS  factor 
is  then  multiplied  by  the  LRU  unit  spares  acquisition  cost  to  provide  the  estimated 
LSCS  dollars: 


LSCS  (Factor)  x  Unit  Cost  =  LSCS 

Removals  per  month  greater  than  one  are  almost  virtual  candidates,  however, 
caution  and  engineering  judgement  should  be  exercised  when  the  age  of  equipment 
starts  to  exceed  10  years.  Exceptions  are  for  units  that  are  completely  overhauled; 
where  the  chassis  may  exceed  10  years,  but  the  lower  level  SRUs  and  assemblies  are 
new  or  refurbished. 


4.4.2  Test  Implementation  Costs 

Test  implementation  costs  consist  of  non-recurring  Amortized  Facility  Cost 
(AFC)  and  the  recurring  Direct  Test  Cost  (DTC)  and  repair.  These  costs  can  nomi¬ 
nally  run  between  7°  and  9°o  of  the  Unit  Cost  (UC),  AFC  +  DTC  =  (7-9°)  •  UC,  and 
are  dependent  upon  such  criteria  as: 

•  Test  chamber  and  equipment  cost 

•  Test  duration 

•  Number  of  units  in  flow 

•  Number  of  test  programs  being  managed  by  AFC 

•  Cost  of  labor  to  perform  tests 

•  Cost  of  repairs  of  failures  encountered  during  test 

•  Cost  of  spares  to  support  repair 

•  Cost  of  additional  support  equipment  to  support  functional  testing. 

4. 4. 2.1  Amortized  Facility  Cost  (AFC)  -  The  non-recurring  amortized  facility  cost  is 
principally  driven  by  the  cost  of  test  equipment  and  set-up  and  effective  test  man¬ 
agement  to  maximize  facility  use  within  the  constraints  of  test  duration  and  unit  flow 
capacity.  This  cost  may  be  expressed  as: 


AFC 

where: 

t 
T 
n 
N 
c 

K 

Reducing  the  number  of  test  equipment  set-ups  (temperature  chambers, 

vibration  equipment,  and  peripherals)  to  optimize  the  test  unit  flow  is  a  major 

impact.  Factors  to  be  considered  include: 

•  Vibration  equipment  is  only  required  approximately  10  minutes  out  of  every 

40  hours  of  testing,  representing  only  8°0  of  the  cycle  time,  and  would 

therefore  require  perhaps  one  vibration  system  for  every  five  to  ten 
temperature  chambers 


v  J  ^ 1  -  Cost  per  Test  Unit  (LRU/SRU) 

(T)  (N)  (K) 

=  test  duration  required  per  unit 

=  number  of  available  test  hours  per  month  per  set  of  test  equipment 
=  LRUs/SRUs  per  month  to  be  tested 
=  total  number  of  LRUs/SRUs  to  be  tested 

=  cost  per  test  equipment  set-up  (temperature,  chamber,  and  vibration 
equipment) 

=  number  of  different  test  programs  to  be  implemented. 


•  Vibration  equipment  normally  represents  approximately  65°o  of  a  single  test 
set-up  cost.  Therefore  minimizing  the  number  of  vibration  systems  is 
significant  in  any  amortized  facility  set-up 

•  Test  fixtures  and  mounting  devices  should  be  as  universal  as  possible. 
Requiring  new  set-ups  for  each  different  type  of  LRU/SRU  to  be  tested  will 
be  costly  and  create  delays  for  set-up. 

4. 4. 2. 2  Direct  Test  &  Repair  Cost  (PTC)  -  Recurring  test  and  repair  costs,  or  the 
costs  to  physically  perform  the  screening  tests,  is  based  on  the  repair  per  unit  as  a 
function  of  the  logistics  support  cost  of  the  LRU/SRU  to  be  tested.  Recurring  costs 
include: 

•  Labor  to  perform  tests 

•  Labor  to  perform  functional  tests 

•  Repairs  for  failures  encountered  during  test 

•  Spares  to  support  repair 

•  Shipping  and  handling  test  units. 

These  are  costs  that  can  be  developed  as  a  function  of  the  logistic  support  cost  for 
the  unit  under  test.  Taking  the  logistic  support  cost  of  the  unit  as  developed  in 
Subsection  4.4.1  and  dividing  by  the  total  expected  number  of  repairs  over  the  life 
cycle  (as  a  function  of  unit  utilization,  expected  service  life,  and  operational  failure 
rate)  provides  the  average  cost  per  repair. 

The  number  of  failures  or  repairs  to  be  encountered  during  the  testing  is  a 
function  of  the  process  average  y  as  defined  in  Subsection  4.2.3.  This  would  be 
the  average  number  of  defects  that  can  be  expected  per  unit.  The  direct  test  cost 
therefore  is  expressed  as: 

T  SC 

DTC  =  - — - x  y  =  cost/unit  tested 

Total  Service 

Life  Repairs 

Figure  C-15  provides  the  DTC  as  a  function  of  the  repair  cost/unit  and  varying 
process  averages. 

4.4.3  Comparative  ROI 

Comparative  ROI  will  provide  the  indications  for  economic  candidate  selection 
when  all  the  variables  of  age,  test  duration,  level  of  test,  unit  flow,  rate  improve¬ 
ment,  etc  have  been  considered.  The  ROI  outcome  in  each  case  should  be  at  least 
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Figure  C-15-  Direct  test  cost  as  a  function  of  cost  per  repair  cost  &  process  average  (ji). 
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33-1/3"  or  greater  to  support  cost  justification.  Table  C-l  and  Fig.  C-16  provide 
the  typical  comparative  analysis  illustration  to  support  justification  of  economic 
selection  of  ranked  LRUs.  Estimates  of  potential  removal  rate  improvement  (MFHBM 
without  ESS  vs  MFHBM  w/ESS)  provide  the  basis  of  logistic  support  savings.  This 
can  be  accomplished  parametrically  using  Fig.  C-15  which  projects  up  to  a  2:1 
potential  improvement  in  removal  rate,  or  by  projecting  the  outcome  as  a  function  of 
the  bad  actor  distribution  as  defined  in  Subsection  1.2. 3. 4.  The  graphic  compari¬ 
sons  (Fig.  C-16)  assess  the  variation  in  the  ROI  as  a  function  of  the  efficiency  of 
the  test  program,  that  is,  the  percentage  of  defects  that  are  reduced  and  the  test 
duration.  The  graphics  show  defect  yield  rates  of  100";  50°u  and  25”,,  considering 
the  test  duration  of  120  or  40  hours,  which  are  the  extremes  per  MIL-STD-21G4(ES) , 
as  well  as  the  age  of  the  LRU  at  time  of  testing.  Irrespective  of  test  duration,  as 
the  potential  gain  in  removal  rate  is  reduced,  the  number  of  economically  viable  units 
significantly  reduces  (ROI  drops  below  33",).  The  most  economical  combination  is 
produced  by  the  reduced  test  duration  with  a  reduction  in  the  number  of  units  to  be 
tested,  while  retaining  a  high  potential  for  defect  reduction,  such  as  with  the  case 
with  the  selection  of  bad  actors.  Optimally  the  selection  should  consider: 

•  High  removal  rate  LRU  contribution,  within  the  top  25  ranking  of  the  weapon 
system  (lower  levels  are  not  cost  effective) 

-  SRUs  drawn  from  these  LRUs  for  testing  should  represent  at  least  25°  of 
the  total  removal  rate/oi  repair  rate  of  the  LRU 

-  Lower  levels  of  assembly  should  not  be  considered  for  field  selection 
(these  should  be  tested  as  procured) 

•  Select  LRU  testing  priority  based  on  bad  actor  selection  criteria.  This 
offers  the  opportunity  to  test  the  least  number  of  units  and  achieve  the  op¬ 
timal  improvement  level  which  is  economically  the  most  significant 

•  Equipment  with  over  10  years  of  service  life  at  the  time  of  evaluation  offers 
little  in  the  way  of  cost  benefit,  unless: 

Removal  rate  impact  is  significant,  greater  than  25°  of  the  weapon  system 
removal  rate 

Unit  is  being  overhauled  for  service  life  extension.  Note  that  replacement 
by  new  unit  (manufacturer  furnished)  may  be  more  cost  effective 

•  For  field  screened  hardware,  eliminate  the  failure  free  portion  of  test  as 
recommended  in  Subsection  4.3.  This  will  reduce  the  test  time  by  at  least 
half  and  offers  the  opportunity  to  test  more  units  in  a  shorter  span  of  time. 
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TABLE  C-1.  ESS  projected  cost  savings  &  ROI 
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OF  UNITS  TESTED  HAVE  DEFECTS  26%  OF  UNITS  TESTED  HAVE  DEFECTS 


5  -  DATA  REQUIREMENTS 


In  support  of  selection  process  criteria,  analysis,  cost  planning,  and  test  plan 
development,  the  following  equipment  characteristic  data  field  data  management  is 
required. 

5.1  EQUIPMENT  CHARACTERISTICS  DATA 

•  Specified  or  Predicted  MTBF  -  includes  any  operational  or  logistic  goals  that 
may  be  defined 

•  Environmental  Design  Levels  -  operational  temperature  and  vibration  levels  to 
support  assessment  of  tailoring  requirements  and  insure  that  ESS  levels  are 
not  design  excessive 

•  Functional  Performance  Characteristic  -  to  insure  adequate  functional  per¬ 
formance  integrity  during  screening,  including  test  point  and  support  equip¬ 
ment  interface  requirements 

•  Level  of  Repair  Requirements  -  to  identify  maintenance  policy  planning  in¬ 
cluding: 

-  Repairability  or  non-repairability 

-  Expected  service  life 

-  Time  to  overhaul  criteria 

-  Level  of  assembly  definition 

-  Logistic  planning 

•  Configuration  &  Age  Data  -  to  define  system  make-up  and  hardware  applica¬ 
tion,  identify: 

-  Average  number  of  units  required  per  system  to  define  full  mission  status 

-  Configuration  (if  units  are  multisystem  oriented,  i.e.,  can  be  used  in 
F15,  B52,  etc) 

-  Number,  age,  and  interchangeability  of  upgrades 

-  LRU  serial  number  blocking  (by  upgrade,  if  possible) 

-  Average  field  service  age  of  equipment  to  be  considered  for  test 

•  Unit  Spares  Acquisition  Cost  -  average  unit  spares  acquisition  price  of  unit 
in  current  year  dollars 
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•  ESS  Status  -  if  screened  at  a  manufacturer's  facility,  as  part  of  acceptance, 
provide: 

-  Tested  to  what  specification  or  standard 

-  Environments  applied  and  levels 

-  Cycle  durations 

-  How  applied,  100°  sampling,  random,  etc 

-  Identifiable  serial  numbers 

•  Warranty  Criteria  -  to  define  potential  limitation  of  field  ESS  and  repair 
implementation,  as  well  as  possibility  to  use  warranty  to  have  suppliers  pro¬ 
vide  ESS  services  and  incentives. 

2  MOPAS/AFDQ56  DATABASE  REQUIREMENTS 

In  support  of  candidate  selection  and  monitoring  in  the  field  maintenance  en- 
ronment,  maintenance  removals  and  repair  data  should  be  reduced,  otherwise  pro- 
rammed,  and  managed  to  provide: 

•  Ranking  LRUs  in  WUC  categories  5,  6,  and  7;  by  total  removals  (Type  1  + 
Type  2  +  Type  6).  Note:  this  should  exclude  all  800  series  How  Mai  Codes 

•  LRU  serial  number  ranking  by  total  removal 

•  Number  of  weapon  systems  reporting  per  period 

•  Number  of  serialized  LRUs  reporting  per  period 

•  Subcategorization  of  Type  1  actions 

-  With  Parts  -  with  completion  of  "H"  and  "P"  card  data 

-  Without  Parts 

•  One  year  (12  month)  moving  process  average  y  (computed  per  Subsection 
4.1.3);  and  distinguishing  between  y^  and  y^ . 

•  SRU  serialization  or  identification  system 

•  ESS  action  taken  codes  for  LRUs  previously  tested 

•  Coding  system  for  identifying  SRU  units  tested. 


6  -  IMPLEMENTATION  OPTIONS 


Once  hardware  types  and  levels  have  been  selected,  options  for  hardware 
collection,  test  management,  and  test  locations  should  be  as  generic  and  standard  as 
practical  for  the  category  of  equipment  selected.  It  should  be  remembered  that  the 
objective  is  to  test  the  equipment  only  once,  and  to  limit  testing  to  only  those  units 
that  offer  the  highest  potential  for  reliability  and  readiness  improvement  at  a 
reasonable  ROI.  This  can  only  be  accomplished  by: 

•  Minimizing  set-up  and  facility  amortization  costs  by  diversification  and 
planning  of  facilities  to  handle  several  equipment  types 

•  Minimizing  cost  and  demands  for  additional  and  specialized  support  equipment 

•  Maximizing  resources,  material,  and  skills  for  rapid  repair  capability  to 
preclude  excessive  testing,  handling,  and  test  interruption  for  lack  of 
spares,  materials,  and  poor  repair  practices 

•  Centralizing  technical  skills  both  for  test  and  equipment  under  test  to  insure 
quick  problem  corrective  action  response,  as  well  as  standardization  of  test 
procedures,  functional  test  parameters,  and  trouble-shooting  methodology. 

The  option  considerations  it.clude  the  following. 

6.1  TESTING  HARDWARE  ON  AN  "AS  REPAIRED"  BASIS 

This  is  screening  performed  on  hardware  only  afte^  it  has  been  through  the 
repair  and  return  to  inventory  cycle.  The  testing  becomes  a  one  time  extension  of 
the  repair  cycle  and  offers  the  best  cost  options  since  it  handles  the  equipment  as  it 
is  in  flow,  minimizing  the  added  support  costs  of  the  ESS  test  as  well  as  the  impact 
on  hardware  availability  and  turnaround  time. 

This  should  be  considered  when: 

•  It  is  concluded  that  a  total  population  of  units  should  be  tested.  In-service 
units  that  have  not  failed  would  not  be-  tested 

•  Testing  low  production  density  systems,  (e.g.,  ground  equipment,  radars, 
etc)  so  as  to  insure  that  equipment  is  always  on-line 


•  Testing  of  repairable  SRUs  and  lower  levels  of  assembly  that  are  identified 
as  candidates.  This  will  require  a  controlled  identification  system  to  identify 
previously  tested  units.  In-service  units  that  do  not  fail  would  not  be 
tested 

•  Testing  bad  actors;  these  could  be  selectively  pulled  in  the  repair  cycle 
based  on  their  identification.  They  would  more  than  likely  be  frequent 
visitors  to  the  repair  facility  in  a  relatively  short  span  of  time.  Non  bad 
actor  units  would  not  be  tested.  This  would,  in  effect,  be  a  form  of 
in-process  sampling  on  a  controlled  basis. 

1  TESTING  HARDWARE  ON  A  "RECALL”  PLAN 

This  is  recalling  hardware  from  the  field  on  a  scheduled  basis,  whether  it  has 
led  or  not.  This  effectively  minimizes  the  test  schedule  impact,  although  test 
>ts  will  accelerate  due  to  the  concentrated  high  demand  on  logistic  material.  This 
pe  of  planning  should  be  considered  when  dealing  with: 

•  High  frequency  problem  units,  preferably  bad  actors 

•  Units  or  systems  that  are  scheduled  for  overhaul 

•  High  readiness  impact  items  that  require  scheduled  inspections  and  tests, 
and  require  as  short  a  downtime  as  possible  because  of  demands 

•  High  cost  or  high  tech  items  where  the  set-up  costs  and  planning  are  a 
problem. 

3  TESTING  HARDWARE  FROM  "READY  FOR  INVENTORY”  STOCK 

This  is  the  pulling  of  previously  repaired  and  accepted  hardware  that  has  been 
turned  to  stock  and  is  awaiting  further  supply  disposition.  The  major  impact  of 
is  is  the  logistic  support  effect  on  existing  field  hardware.  Pulling  of  good 
rdware  from  stock  and  environmentally  testing  them  will  probably  require 
ditional  repairs,  creating  logistic  shortages  and  affecting  system  readiness.  It 
:o  accelerates  test  costs,  as  well  as  field  logistic  costs.  For  these  reasons,  it 
ould  only  be  considered  on  a  planned  scheduled  basis  to  insure  availability  of 
jcks  to  support  operationally  deployed  systems. 

This  would  better  be  applied  to  lower  levels  of  assembly  that  are  stockpiled  and 
e  designated  for  100°  screening.  This  may  be  considered  to  upgrade  stocks  to  be 
mpatible  with  new  vendor  delivered  stocks  that  are  screened  at  the  manufacturer's 
cility . 


6.4  ALC  CENTRALIZED  TEST  FACILITY  VS  INTERMEDIATE  MAINTENANCE  TEST 


;  FACILITIES 

For  reasons  enumerated  in  the  introduction  of  this  section,  the  optimal  cost  and 
technically  effective  approach  to  handling  the  enormous  and  diverse  complexities  of 
the  equipment  type,  state  of  the  art  configuration,  and  performance  criteria  is  to 
establish  centralized  ALCs  that  can  handle  the  facilities,  testing,  and  repair 
technology,  and  flow  of  the  hardware.  The  use  of  intermediate  test  facilities  (e.g., 
at  tactical  airbases)  creates  a  massive  decentralization  control  problem  both  in  test 
technology  and  in  identifying  what  equipment  should  be  tested,  as  well  as  what  has 
been  tested.  The  effectiveness  of  ESS  is  not  only  the  environment,  but  the  planned 
process  control  that  goes  with  it  to  insure  end  item  continuity  which  bottom-lines  the 
ROI  (profit  margin  to  a  manufacturer). 

The  immediate  problem  is  that  the  ALCs,  in  many  cases,  cannot  handle  large 
volumes  of  LRU  level  hardware;  they  are  more  geared  to  SRU  level  repair  and 
testing.  This  requires  some  re-thinking  of  Air  Force  material  maintenance 
management  policies  and  planning,  at  least  on  a  limited  basis. 

6.5  CONTRACTING  FOR  FACILITIES 

The  contracting  of  test  facilities  on  a  competitive  basis  can  only  be  effective  if 
it  is  adjunct  to  the  requirements  and  control  of  the  ALC,  or  is  actually  the 
manufacturer  of  the  equipment  to  be  tested.  The  contracted  facility  should  not  be 
purely  a  test  laboratory  but  actually  become  a  process  control  arm  of  the  ALC, 
requiring  the  establishment  of  formal  test  planning  and  repair,  and  providing  the 
skills  and  capability  necessary  to  restore  the  equipment  tested  to  a  ready  for 
inventory  condition. 
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